Dataset

We curate a novel multi-view multi-sequence dataset for single-object system identification. We have 10 different object geometries: Egg, Cushion, Apple, Bread, Cream, Barrel, Potato, Pawn, Banana, and Mushroom. These objects are evenly distributed over 5 materials: Elastic, Elastoplastic, Liquid, Sand, and Snow. For each object, 10 sequences of that object are simulated using the Genesis World [2] physics engine, each with a different initial location, rotation, and velocity. Each sequence has 11 total camera views around the object.

Experimental Setup

We train our models and baselines with 1, 2, and 3 total sequences, then test them on 2 new unseen sequences. We initially wanted to train on more sequences, but training on 4 sequences already consumes >50 GB of GPU memory for our baseline !

Here are some metrics tested against our main baseline, Gaussian Informed Continuum [1]:

Our model generally performs equal to or better than the baseline on three different metrics: Chamfer Distance, which measures the distance between ground truth and predicted point clouds, and PSNR and SSIM, which measure the image loss on rendered ground truth and rendered predicted 3D Isotropic Gaussians.

Multi-Object System Indentification

In order to create a benchmark for the multi-object setting, we reuse assets from the multi-sequence dataset to generate new multi-view sequences of multi-object collisions.

We evaluate our method against an adapted version of OmniPhysGS [3] and CoupNeRF [4]. Although both methods do not directly solve our task, they are able to handle multi-object interactions, so we adapt/reproduce their works to solve our task. Our method achieves better performance than these baselines.

Here are some qualitative comparisons:

We also evaluate each method using PSNR and SSIM for 2D rendered images, and Chamfer Distance and Earth Mover’s Distance for 3D points. Quantitatively, we outperform baseline methods.

References

[1] Junhao Cai, Yuji Yang, Weihao Yuan, Yisheng He, Zilong Dong, Liefeng Bo, Hui Cheng, and Qifeng Chen. Gic: Gaussian-informed continuum for physical property identification and simulation. arXiv preprint arXiv:2406.14927, 2024.

[2] Genesis Authors. Genesis: A universal and generative physics engine for robotics and beyond, December 2024

[3] Yuchen Lin, Chenguo Lin, Jianjin Xu, and Yadong Mu. Omniphysgs: 3d constitutive gaussians for general physics-based dynamics generation. arXiv preprint arXiv:2501.18982, 2025.

[4] Jin Li, Yang Gao, Song Wenfeng, Yacong Li, Shuai li, Aimin Hao, and Hong Qin. Coupnerf:
Property-aware neural radiance fields for multi-material coupled scenario reconstruction. Computer Graphics Forum, 43, 10 2024a. doi: 10.1111/cgf.15208.