3D Diffusion Policy (DP3)<\/a><\/code><\/strong><\/em> combines point cloud observations with a diffusion-based action prediction model, making it a strong low-level execution backbone for tasks that require precise geometric control.<\/p>\n\n\n\n![\"\"](\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf18\/wp-content\/uploads\/sites\/160\/2026\/05\/DP3-1024x425.png\")
<\/figure>\n\n\n\nIn this project, we use a DP3-style low-level policy, but we add explicit goal conditioning. <\/strong>The low-level policy receives the current scene, the observed hand configuration, the predicted goal hand configuration, and the robot state. It then predicts robot actions through a diffusion denoising process.<\/p>\n\n\n\nGoal-Conditioned Hierarchical Policies<\/h3>\n\n\n\n
ArticuBot<\/a><\/code><\/em><\/strong> demonstrates a hierarchical, goal-conditioned policy for articulated object manipulation with parallel-jaw grippers. Its high-level policy predicts end-effector goal points from the current observation, and its low-level policy executes actions conditioned on the predicted goal. This architecture is effective because it separates interaction-goal prediction from action execution.<\/p>\n\n\n\n![\"\"](\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf18\/wp-content\/uploads\/sites\/160\/2026\/05\/image-3.png\")
<\/figure>\n\n\n\nOur project follows this high-level \/ low-level structure, but targets dexterous hands rather than parallel-jaw grippers.<\/strong> This makes the goal design more challenging. A parallel-jaw gripper can often be represented by a few gripper points, while a dexterous hand needs to represent multiple fingers, palm placement, and richer contact geometry.<\/p>\n\n\n\nDexterous Grasp Priors<\/h3>\n\n\n\n
Large-scale dexterous grasp datasets provide useful priors for understanding hand-object interaction. Dexonomy<\/a><\/strong> <\/code><\/em>provides a taxonomy of dexterous grasp types, helping organize the diversity of hand configurations. GraspXL<\/a><\/code><\/em><\/strong> provides large-scale grasping motions for diverse objects, offering another source of dexterous hand-object interaction data.<\/p>\n\n\n\n![\"\"](\"https:\/\/mscvprojects.ri.cmu.edu\/2026teamf18\/wp-content\/uploads\/sites\/160\/2026\/05\/dexonomy_smaller-1024x473.png\")
<\/figure>\n\n\n\nThese datasets motivate our focus on sparse but expressive hand representations.<\/strong> Rather than representing a goal only as an object pose or a single robot end-effector position, we represent the desired dexterous hand configuration using 3D points on fingers, finger links, and the palm.<\/p>\n\n\n\n<\/p>\n\n\n\n
References<\/h5>\n\n\n\n[1] Ze, Yanjie, et al. \u201c3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations.\u201d Proceedings of Robotics: Science and Systems<\/em>, 2024. https:\/\/3d-diffusion-policy.github.io\/<\/p>\n[2] Wang, Yufei, et al. \u201cArticuBot: Learning Universal Articulated Object Manipulation Policy via Large Scale Simulation.\u201d Proceedings of Robotics: Science and Systems<\/em>, 2025. https:\/\/articubot.github.io\/<\/p>\n