Topology-Aware Gaussian Splatting for Dynamic Mesh Modeling and Tracking

1Beijing Engineering Research Center of Mixed Reality and Advanced Display,School of Optics and Photonics, Beijing Institute of Technology 2Soul Shell Technology Co., Ltd.
MY ALT TEXT
MY ALT TEXT

Pipeline figure

Abstract

We propose a topology-aware dynamic reconstruction framework based on 3D Gaussian splatting, addressing the challenges of topology-consistent mesh reconstruction and tracking in dynamic scenes. We design a Gaussian topology structure to represent spatial relationships among Gaussians and propose a densification and pruning strategy that preserves the manifold structure. By initializing Gaussians from an initial mesh reconstruction and incorporating multiple temporal consistency regularizations, we achieve stable dynamic deformation of Gaussian parameters. In addition, differentiable mesh rasterization is introduced to further improve the quality of the reconstructed mesh. Our approach produces both same-topology Gaussian sequences and mesh sequences, while enabling high-precision 3D keypoint tracking. This provides a high-fidelity and cost-effective solution for downstream tasks such as animation production. Extensive experiments demonstrate that our method outperforms existing approaches in both mesh-reconstruction accuracy and tracking precision.

Tracking results

Modeling results