{"id":18,"date":"2023-05-07T23:00:24","date_gmt":"2023-05-07T23:00:24","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/?p=18"},"modified":"2023-05-07T23:21:34","modified_gmt":"2023-05-07T23:21:34","slug":"spring-progress","status":"publish","type":"post","link":"https:\/\/mscvprojects.ri.cmu.edu\/f23team8\/2023\/05\/07\/spring-progress\/","title":{"rendered":"Spring Progress"},"content":{"rendered":"\n

Here is what we have done in the spring of 2023. Our presentation for the paper survey is here<\/a>.<\/p>\n\n\n\n

Survey over Existing Datasets<\/p>\n\n\n\n

GMU-Kitchen<\/a> is a dataset consisting of 9 RGB-D kitchen video sequences with object annotations. It lacks some details, and does not contain any human activity.<\/p>\n\n\n\n

\"GMU-Kitchen\"<\/figure>\n\n\n\n

ScanNet<\/a> is a dataset of over 1600 indoor scenes with detailed annotations.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

There are also a couple of other datasets, but in they contain either only static scenes or no 3D information. A table here shows the difference:<\/p>\n\n\n\n

Dataset<\/td>3D Information<\/td>Human Activity<\/td>Kitchen<\/td><\/tr>
Epic Kitchen<\/td>\u2718<\/td>\u2714<\/td>\u2714<\/td><\/tr>
GMU Kitchen<\/td>\u2714<\/td>\u2718<\/td>\u2714<\/td><\/tr>
MSR Action3D<\/td>\u2714<\/td>\u2714<\/td>\u2718<\/td><\/tr>
ScanNet<\/td>\u2714<\/td>\u2718<\/td>\u2718<\/td><\/tr><\/tbody><\/table>
Comparison of Some Existing Datasets<\/figcaption><\/figure>\n\n\n\n

Camera Setup<\/p>\n\n\n\n

We have built a miniature synchronized multi-camera capture system in our lab and a digital twin in Mujoco to test our calibration and scene understanding algorithms; we plan to replicate the system in a real kitchen later.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

We then use AprilTags to define our world coordinates and calibrate camera extrinsics.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Scene Segmentation<\/p>\n\n\n\n