Project summary

Motivation

• The importance of hand in (virtual) communications.
• More technically challenging compared to body or face.

Problem

• Task: Multiview 3D pose estimation
• Data: Sequences of hand images captured from multiple cameras in multiple views

Solution

• Extract feature and detect keypoints in 2D image
• Lifting 2D keypoints to 3D
• Epipolar geometry to improve 2D features

Documents:

Demo:

https://drive.google.com/open?id=1m8D4FmD_LOND6eG4BMoQjFLf1AJEDWSE

Capstone presentation:

https://drive.google.com/open?id=1gdyv2MmLz65pv24CxRmHtql_uxXVlupv

Mid-semester presentation:

https://drive.google.com/open?id=1ShSedeV_4D9sFfbiuQYGJ8PltNS6XcQS

Final presentation:

https://docs.google.com/presentation/d/1il_ufJcrc46Nle44Ffs73PLAobKcpAKSlNvGhNI8hoQ/edit?usp=sharing

Project summary

• Motivation
  • The importance of hand in (virtual) communications.
  • More Technically challenging compared to body or face.

• Problem
  • Task: Two-hand pose estimation and mesh tracking
  • Data: Sequences of hand images captured from multiple cameras in multiple views

• Solution
  • Extract feature and detect keypoints in 2D images
  • 2D -> 3D:
    • Use neural network to learn the prior
    • Aggregate multiview 2D features into 3D
  • Structural learning and temporal consistency?

Team

Yihui He: http://yihui-he.github.io/

Rui Yan:

Documents:

Demo:

https://drive.google.com/open?id=1s72aBYBsyeCaBc4_qMysVmYueZ2_kOCe

Slides:

https://docs.google.com/presentation/d/1il_ufJcrc46Nle44Ffs73PLAobKcpAKSlNvGhNI8hoQ/edit?usp=sharing