Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Dynamic Scenes
Authors: Isabella Liu, Hao Su, Xiaolong Wang
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | During the evaluation on different datasets, DG-Mesh provides significantly better mesh reconstruction and rendering than baselines. Our method provides significantly better mesh reconstruction and rendering than baselines. 4 EXPERIMENTS 4.4 ABLATION STUDY |
| Researcher Affiliation | Academia | Isabella Liu, Hao Su , Xiaolong Wang UC San Diego, Equal advising |
| Pseudocode | Yes | A GAUSSIAN-MESH ANCHORING DETAILS ... Algorithm 1: Gaussian-Mesh Anchoring |
| Open Source Code | Yes | Codes and data are publicly available at https://github.com/Isabella98Liu/DG-Mesh. |
| Open Datasets | Yes | Codes and data are publicly available at https://github.com/Isabella98Liu/DG-Mesh. We evaluate our method on the D-Ne RF synthetic dataset and provide the visualization and comparison with other baselines in Figure 5. Since the D-Ne RF (Pumarola et al., 2021) dataset does not provide mesh ground truth information, we rendered a synthetic dataset containing six dynamic scenes to compare the mesh reconstruction quality quantitatively. Each scene includes 200 frames of a moving object with the ground truth camera parameters and images, as well as the ground truth mesh under each time frame. ... For real data evaluation, we run our method on the Nerfies dataset (Park et al., 2021a) , the Dycheck s dataset (Gao et al., 2022) and the Unbiased4D dataset (Johnson et al., 2023), all of which contains monocular videos of everyday deformable objects captured using handheld cameras. To demonstrate the adaptability of DG-Mesh to different dynamic capturing setups, we also test our method on the Neural Actor (Liu et al., 2021) dataset, which features multi-view videos of moving humans. |
| Dataset Splits | No | The paper mentions evaluating on various datasets such as D-Ne RF and DG-Mesh datasets, but does not provide specific details on how these datasets were split into training, validation, or test sets, nor does it refer to standard predefined splits. For example, it says 'Each scene includes 200 frames of a moving object' for the DG-Mesh dataset, but not how these frames are divided for experimental purposes. |
| Hardware Specification | Yes | Our model was trained for a total of 50,000 iterations on a single RTX 3090Ti. |
| Software Dependencies | No | The paper mentions using '3D Gaussian Splatting (Kerbl et al., 2023)' and 'Nvdiffrast (Laine et al., 2020)' but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | Our model was trained for a total of 50,000 iterations on a single RTX 3090Ti. To better initialize deformable 3D Gaussians, we first trained the canonical Gaussians for 3k iterations while keeping the forward and backward deformation network fixed. This helped to retain relatively stable positions and shapes of 3D Gaussians under the canonical space. After 5k iterations, we introduce the DPSR and differentiable Marching Cubes to extract the mesh geometry from the Gaussian points. We perform Gaussian-Mesh Anchoring every 100 iteration during training. ... both the forward and backward deformation networks are designed as multi-layer perceptrons (MLPs) with a depth of D = 8 layers and hidden layers of dimensionality W = 256. ... Specifically, we set the encoding parameter k = 10 for the positions x of the 3D Gaussians and k = 6 for the time labels t, enhancing the network s ability to capture high-frequency variations in both space and time. ... As shown in Table 3, when wlap = 1000, the network produces the highest mesh rendering quality. With appropriate laplacian regularization, our method recovers a smoother surface. ... Based on the ablation results, we set the anchoring interval to 100, as it represents a balanced trade-off between maintaining geometry quality and achieving optimal rendering performance. |