Implementation
Its implementation requires only a set of differentiable building blocks that can work as a drop-in replacement in existing mesh-processing architectures.
We propose "Adaptive-Spiral", a new 3D operator that adapts its weights to each mesh and learns vertex-adaptive and mesh-adaptive spiral sequences of variable lengths.
The success of deep learning models on structured data has generated significant interest in extending their application to non-Euclidean domains.
In this work, we introduce a novel intrinsic operator suitable for representation learning on 3D meshes. Our operator is specifically tailored to adapt its behavior to the irregular structure of the underlying graph and effectively utilizes its long-range dependencies, while at the same time ensuring computational efficiency and ease of optimization.
In particular, inspired by the framework of Spiral Convolution, which extracts and transforms the vertices in the 3D mesh following a local spiral ordering, we propose a general operator that dynamically adjusts the length of the spiral trajectory and the parameters of the transformation for each processed vertex and mesh.
Then, we use polyadic decomposition to factorize its dense weight tensor into a sequence of lighter linear layers that separately process features and vertices information, hence significantly reducing the computational complexity without introducing any stringent inductive biases. Notably, we leverage dynamic gating to achieve spatial adaptivity and induce global reasoning with constant time complexity benefitting from an efficient dynamic pooling mechanism based on Summed-Area-tables.
Used as a drop-in replacement on existing architectures for shape correspondence our operator significantly improves the performance-efficiency trade-off, and in 3D shape generation with morphable models achieves state-of-the-art performance with a three-fold reduction in the number of parameters required.
Its implementation requires only a set of differentiable building blocks that can work as a drop-in replacement in existing mesh-processing architectures.
An Adaptive-Spiral 3D Morphable Model generates new realistic representations, and can transition from one expression to another via latent-space interpolation.
Mouth Extreme
Cheeks In
Eyebrow
High Smile
Bare Teeth
Mouth Down
Plotting its kernels via t-SNE highlights their adaptive behavior. Kernels are Vertex-Adaptive: Weights processing the same vertex are grouped in distinct clusters. Kernels are Mesh-Adaptive: Inside each group, a different response can be identified for every mesh.
Adaptive-Spiral predicts the length of the spiral for each mesh and vertex. Moreover, it automatically learns to enlarge the length of the spiral to learn hierarchical representations (receptive field increase from level 1 to level 3). Compared to our method, the original Spiral Convolution processing is limited to a small spiral of fixed length.
@article{babiloni2023adaptive,
author = {Babiloni, Francesca and Maggioni, Matteo and Tanay, Thomas and Deng, Jiankang and Leonardis, Ales and Zafeiriou, Stefanos},
title = {Adaptive Spiral Layers for Efficient 3D Representation Learning on Meshes},
journal = {ICCV},
year = {2023},
}