Object inspection through registration of its CAD model to an image stream in real-time.


The Aircraft Maintenance, Repair, and Overhaul(MRO) market is valued at $87.01 billion in 2021 and is projected to grow to $118.96 billion by 2028. Currently, aircraft are manually inspected by human inspectors. This is a time-consuming process since human inspectors are limited in their ability to inspect all areas of a large aircraft. The inspection process also presents safety hazards for the inspectors involved.

The goal of this project is to build an automated system for high-fidelity aircraft inspection by transferring texture seen in a video feed of an aircraft onto a CAD model of the aircraft in real time. The ability to automate the inspection process allows for better coverage of the aircraft while being significantly faster and safer than manual inspection.

Problem Statement

Build an automated system for high-fidelity inspection of aircraft by registering CAD models of aircraft to an image stream in real time.

Proposed Solution

The proposed solution consists of two parts:

  • First, the absolute pose of the aircraft in each frame is predicted using a pose detection network
  • Next, the predicted pose is used to transfer the texture from the RGB image to the untextured CAD model

The two steps are shown in the figures below:

(a) Pose Estimation

(b) Texture Transfer