We have built a miniature synchronized multi-camera capture system in our lab for the scope of this project – this system can be replicated in a real kitchen later.
We also developed a digital twin in Mujoco to test our calibration and scene understanding algorithms.
- Intrinsics are calibrated by the factory for Azure Kinect cameras
- Extrinsics are estimated by optimizing Procrustes with corresponding points
- Anchor points are obtained by AprilTag detectors