Dataset
We curate a novel multi-view multi-sequence dataset for single-object system identification. We have 10 different object geometries: Egg, Cushion, Apple, Bread, Cream, Barrel, Potato, Pawn, Banana, and Mushroom. These objects are evenly distributed over 5 materials: Elastic, Elastoplastic, Liquid, Sand, and Snow. For each object, 10 sequences of that object are simulated using the Genesis World [2] physics engine, each with a different initial location, rotation, and velocity. Each sequence has 11 total camera views around the object.
Experimental Setup
We train our models and baselines with 1, 2, and 3 total sequences, then test them on 2 new unseen sequences. We initially wanted to train on more sequences, but training on 4 sequences already consumes >50 GB of GPU memory for our baseline !
Here are some metrics tested against our main baseline, Gaussian Informed Continuum [1]:
Our model generally performs equal to or better than the baseline on three different metrics: Chamfer Distance, which measures the distance between ground truth and predicted point clouds, and PSNR and SSIM, which measure the image loss on rendered ground truth and rendered predicted 3D Isotropic Gaussians.
References
[1] Junhao Cai, Yuji Yang, Weihao Yuan, Yisheng He, Zilong Dong, Liefeng Bo, Hui Cheng, and Qifeng Chen. Gic: Gaussian-informed continuum for physical property identification and simulation. arXiv preprint arXiv:2406.14927, 2024.
[2] Genesis Authors. Genesis: A universal and generative physics engine for robotics and beyond, December 2024