Realistic human motion in 3D scenes is fundamental to robotics, animation, and digital humans. Yet current methods treat hand-object interaction and full-body scene interaction as separate problems, leaving a critical gap in generating coordinated, realistic whole-body motion.
We aim to combine HOI and HSI for full-body motion with dexterous hand manipulation in 3D scenes.
Our goals are:
- Unified HOI-HSI Generation — Existing HOI models only animate hands, while HSI models do full-body motion with simple actions. We bridge both into a single framework.
- Autonomous Planner — A VLM/LLM breaks complex, long-horizon tasks into autoregressive motion sequences.
- Interactive Control — Users can guide and correct motion using text, waypoints, or direct instructions.
- Diverse Motion Generation — Supporting bimanual interactions and a broader range of objects, beyond the right-hand-only defaults of current methods.
