Data Collection Setup

Camera Setup

We have built a miniature synchronized multi-camera capture system in our lab for the scope of this project – this system can be replicated in a real kitchen later.

We also developed a digital twin in Mujoco to test our calibration and scene understanding algorithms.

Mujoco Setup

Camera Calibration

  • Intrinsics are calibrated by the factory for Azure Kinect cameras
  • Extrinsics are estimated by optimizing Procrustes with corresponding points
  • Anchor points are obtained by AprilTag detectors
AprilTag detectors