Creative and sequential tasks using hands, like origami, welding, or knitting, are difficult for beginners to learn. Traditional methods of instruction, like instruction manuals or videos, do not offer feedback on how well a learner is performing the task. Feedback is crucial in the learning process, as it helps learners understand what they are doing well and what needs improvement. In contrast, modern computer vision systems and projection interfaces can provide real-time feedback to the user, tracking progress and correcting errors as they occur. This is where Origami Sensei comes in! Origami Sensei is a data-driven vision-based system that provides step-by-step instructions and real-time feedback to assist users in creating origami. It is designed to be intuitive and interactive, making it easy for beginners to follow. With Origami Sensei, users can choose from a library of pre-existing designs or create their own origami. The complete system will utilize computer vision models and algorithms to achieve three main functionality:
- Recognizing users’ current step (Spring 2023);
- Providing guidance to move to the next step required to complete the work (Fall 2023);
- Recognizing when the user makes a mistake and giving tailored instructions to correct it (Fall 2023).
With these components, compared to existing forms of origami instructions, we provide a new modality of origami tutorials with an unprecedented level of user-friendly interaction and automation.