---

Related Works

The taxonomy of 4D reconstruction methods can be broadly divided into four categories.

Dynamic Reconstruction and Tracking from Videos

3D Gaussian Splatting (3DGS)¹ and dynamic extensions such as 4DGS², Dynamic 3DGS³, and deformable GS variants (such as SC-GS⁴) augment Gaussians with learned deformation fields or canonical-space representations to capture scene dynamics from video. Monocular reconstruction methods such as Shape of Motion⁵ and Feedforward approaches such as St4rtrack⁶, among several others, predict depth, point maps, scene flow, or Gaussians across time in a single pass. Across all these methods, reconstructions are constrained to the camera’s field of view: unobserved object surfaces remain empty or distorted, and these approaches do not complete the full 360° geometry and appearance of the dynamic object.

Generative 4D Novel View Synthesis

A class of methods leverages video diffusion models conditioned on target camera trajectories to hallucinate novel views. These methods ground generation in observed structure via intermediate representations such as depth, point tracks, or geometry latents. GEN3C⁷ for example, projects input frames into an explicit 3D point cloud and uses it to condition a video diffusion model, while CogNVS⁸ follows a reconstruct, inpaint, then finetune pipeline for dynamic novel-view synthesis from monocular video. While compelling for moderate viewpoint changes, these methods degrade under extreme extrapolation, where large unseen regions must be hallucinated, owing to the scarcity of diverse multi-view video training data.

Feedforward Generative 4D Reconstruction

Rather than generating novel views, another line of work directly predicts complete 4D representations from video in a single forward pass. L4GM⁹ trains a large Gaussian reconstruction model on synthetic multi-view video renderings of animated assets, enabling sub-second video-to-4D reconstruction. ActionMesh¹⁰ extends 3D latent diffusion with a temporal axis and trains on animated assets to produce temporally coherent animated meshes. Motion 3-to-4¹¹ decomposes the problem into static shape generation and motion reconstruction, learning compact motion latents over a canonical mesh and predicting per-frame vertex trajectories via a framewise transformer. The key limitation of these methods is the dependence on synthetic or category-specific 4D assets for training, which are expensive to produce and limited in diversity. Consequently, these models generalize poorly to in-the-wild videos with occlusions, large non-rigid deformations, or novel object categories.

Prior-aided 4D Reconstruction

Given the scarcity of 4D training data and multiview video, a growing body of work builds 4D representations via test-time optimization guided by large-scale 2D or 3D generative priors. One class keeps such a prior continuously in the loop, either as a score-distillation signal over a dynamic Gaussian or NeRF field (such as DreamScene4D¹²), or as spatiotemporally consistent multi-view video supervision from a diffusion model (such as SV4D¹³). A second class uses a prior only to initialize a canonical geometry from categoryspecific templates (such as Banmo¹⁴) or image-to-3D models (such as V2M4¹⁵), then refines with video supervision alone; PAD3R¹⁶, closely related to our work, initializes a canonical 3D model via an image-to-3D prior, trains a personalized pose estimator on its renderings, and uses the resulting pose initialization to guide deformable Gaussian optimization for category-agnostic reconstruction from casual monocular video. Methods keeping the prior in the loop inherit data scarcity issues or suffer from domain gap due to lacking temporal correspondence, while optimizing from prior-initialized geometry remains ill-posed—many plausible motions and appearances can explain the observed video—often yielding degenerate geometry, motion, or appearance in unobserved regions.

---

References

1. Kerbl, B., Kopanas, G., Leimkühler, T., Drettakis, G.: 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics (TOG) (2023) ↩︎
2. Wu, G., Yi, T., Fang, J., Xie, L., Zhang, X., Wei, W., Liu, W., Tian, Q., Wang, X.: 4d gaussian splatting for real-time dynamic scene rendering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2024) ↩︎
3. Luiten, J., Kopanas, G., Leibe, B., Ramanan, D.: Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In: Proceedings of the International Conference on 3D Vision (3DV) (2024) ↩︎
4. Huang, Y.H., Sun, Y.T., Yang, Z., Lyu, X., Cao, Y.P., Qi, X.: Sc-gs: Sparsecontrolled gaussian splatting for editable dynamic scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2024) ↩︎
5. Wang, Q., Ye, V., Gao, H., Zeng, W., Austin, J., Li, Z., Kanazawa, A.: Shape of motion: 4d reconstruction from a single video. In: Proceedings of the International Conference on Computer Vision (ICCV) (2025) ↩︎
6. Feng, H., Zhang, J., Wang, Q., Ye, Y., Yu, P., Black, M.J., Darrell, T., Kanazawa, A.: St4rtrack: Simultaneous 4d reconstruction and tracking in the world. In: Proceedings of the International Conference on Computer Vision (ICCV) (2025) ↩︎
7. Ren, X., Shen, T., Huang, J., Ling, H., Lu, Y., Nimier-David, M., Müller, T., Keller, A., Fidler, S., Gao, J.: Gen3c: 3d-informed world-consistent video generation with precise camera control. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2025) ↩︎
8. Chen, K., Khurana, T., Ramanan, D.: Reconstruct, inpaint, test-time finetune: Dynamic novel-view synthesis from monocular videos. In: Advances in Neural Information Processing Systems (NeurIPS) (2025) ↩︎
9. Ren, J., Xie, K., Mirzaei, A., Liang, H., Zeng, X., Kreis, K., Liu, Z., Torralba, A., Fidler, S., Kim, S.W., Ling, H.: L4gm: Large 4d gaussian reconstruction model. In: Advances in Neural Information Processing Systems (NeurIPS) (December 2024) ↩︎
10. Sabathier, R., Novotny, D., Mitra, N.J., Monnier, T.: Actionmesh: Animated 3d mesh generation with temporal 3d diffusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2026) ↩︎
11. Hongyuan, C., Xingyu, C., Zhang, Y., Zexiang, X., Anpei, C.: Motion 3-to-4: 3d motion reconstruction for 4d synthesis. arXiv (2026) ↩︎
12. Chu, W.H., Ke, L., Fragkiadaki, K.: Dreamscene4d: Dynamic multi-object scene generation from monocular videos. In: Advances in Neural Information Processing Systems (NeurIPS) (2024) ↩︎
13. Xie, Y., Yao, C.H., Voleti, V., Jiang, H., Jampani, V.: SV4D: Dynamic 3d content generation with multi-frame and multi-view consistency. In: Proceedings of the International Conference on Learning Representations (ICLR) (2025) ↩︎
14. Yang, G., Vo, M., Neverova, N., Ramanan, D., Vedaldi, A., Joo, H.: Banmo: Building animatable 3d neural models from many casual videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2022) ↩︎
15. Chen, J., Zhang, B., Tang, X., Wonka, P.: V2m4: 4d mesh animation reconstruction from a single monocular video. In: Proceedings of the International Conference on Computer Vision (ICCV) (2025) ↩︎
16. Liao, T.H., Liu, H., Xu, Y., Ge, S., Yang, G., Huang, J.B.: Pad3r: Pose-aware dynamic 3d reconstruction from casual videos. In: SIGGRAPH Asia (2025) ↩︎