Introduction

Title

Object inspection through registration of its CAD model to an image stream in real-time.

Motivation

Figure: Current and expected market sizes of aircraft MRO market

The Aircraft Maintenance, Repair, and Overhaul(MRO) market is valued at $87.01 billion in 2021 and is projected to grow to $118.96 billion by 2028. Currently, aircraft are manually inspected by human inspectors. This is a time-consuming process since human inspectors are limited in their ability to inspect all areas of a large aircraft. The inspection process also presents safety hazards for the inspectors involved.

The goal of this project is to build an automated system for high-fidelity aircraft inspection by transferring texture seen in a video feed of an aircraft onto a CAD model of the aircraft in real-time. The ability to automate the inspection process allows for better coverage of the aircraft while being significantly faster and safer than manual inspection.

Problem Statement

Build an automated system for high-fidelity inspection of aircraft by registering CAD models of aircraft to an image stream in real-time.

Proposed Solution

The proposed solution consists of two parts:

  • First, the absolute pose of the aircraft in each frame is predicted using a pose detection network
  • Next, the predicted pose is used to transfer the texture from the RGB image to the untextured CAD model

The two steps are shown in the figures below:

Figure: Step 1 of the pipeline that involves pose estimation
Figure: Step 2 of the pipeline that involves texture transfer from RGB image to an untextured CAD model