Recent work in streaming and dynamic 3D Gaussian Splatting has shaped our project from several angles: how to update a scene quickly over time, how to avoid wasting computation on static regions, how to structure Gaussians more efficiently, and how to use motion cues from optical flow rather than relying only on reconstruction loss.<\/p>\n\n\n\n
3DGStream<\/strong> showed that free-viewpoint video can be reconstructed in a streaming, frame-by-frame manner instead of requiring a full video sequence to be trained offline. Its key insight is to avoid directly optimizing all Gaussians at every timestep; instead, it uses a compact Neural Transformation Cache (\u201cNTC\u201d) to predict Gaussian translations and rotations, which reduces per-frame training and storage while maintaining real-time rendering. This is the closest baseline to our setting because it directly targets streamable dynamic scenes<\/p>\n\n\n\n QUEEN<\/strong> builds on the idea of frame-to-frame residual updates, but focuses more heavily on compact encoding. Rather than storing full Gaussian updates, it learns and compresses attribute residuals between consecutive frames using a quantization-sparsity framework. Its most relevant idea for us is using viewspace gradient differences to identify which Gaussians actually belong to dynamic regions, so training and streaming can focus on the parts of the scene that actually change.<\/p>\n\n\n\n Scaffold-GS <\/strong>addresses a different but important issue: raw 3D Gaussian clouds can be redundant and poorly organized. It introduces a sparse grid of anchor points, where each anchor generates local neural Gaussians and predicts their attributes based on viewing direction and distance. This motivates the idea that a more structured Gaussian representation can reduce redundancy while still preserving high-quality rendering<\/p>\n\n\n\n Octree-GS <\/strong>extends this structure further by organizing anchors into a hierarchy of levels of detail. Instead of rendering every Gaussian equally, it dynamically selects the appropriate level based on the view, so distant views can use coarser detail while closer views can access finer structure. Although this work is only for static and not temporal scenes, it is still relevant to our long-term goal of reducing storage and bandwidth by exploiting spatial redundancy in the Gaussian field.<\/p>\n\n\n\n Motion-Aware 3DGS<\/strong> is particularly important for our optical-flow direction. It argues that dynamic 3DGS methods often overlook the motion information already present in 2D video, then uses optical flow to connect pixel-level motion with 3D Gaussian movement. Its use of flow-based motion correspondence, uncertainty-aware flow losses, and dynamic-aware optimization supports our core intuition that optical flow can act as a useful motion prior for faster and more coherent Gaussian updates<\/p>\n\n\n\n Together, these works show strong progress toward streamable dynamic Gaussian reconstruction, but important limitations remain. Per-frame updates can still be too slow for true real-time streaming, compact representations can trade off against visual quality, and many methods still rely heavily on optimization to discover motion rather than using motion cues directly. Our project aims to address this gap by using optical flow as an explicit motion signal, separating dynamic and static regions, and eventually combining motion-aware updates with efficient scene structures such as hash grids and octrees.<\/p>\n\n\n\n <\/p>\n\n\n\n References:<\/strong><\/p>\n\n\n\n <\/p>\n","protected":false},"excerpt":{"rendered":" Recent work in streaming and dynamic 3D Gaussian Splatting has shaped our project from several angles: how to update a scene quickly over time, how to avoid wasting computation on static regions, how to structure Gaussians more efficiently, and how to use motion cues from optical flow rather than relying only on reconstruction loss. 3DGStream … <\/p>\n\n