Method

Overview

We propose CoMODAL, an active learning framework for object detection that uses multiple foundation models cooperatively. Instead of relying on a single model, CoMODAL assigns different foundation models to different stages of the pipeline, including negative image mining, query sampling, and accelerated annotation.

Given an unlabeled image set and a target object class, CoMODAL first identifies reliable negative images, then selects informative query samples from the remaining unlabeled pool, and finally asks the human annotator to verify candidate bounding boxes generated by foundation models.

Negative Image Mining

CoMODAL begins by finding images that confidently do not contain the target class. A committee of vision-language models checks whether each image contains the object of interest. Only images that all committee members agree are negative are added to the negative set.

This step reduces unnecessary annotation effort and provides useful negative supervision for detector training. The design favors precision: uncertain cases are kept in the unlabeled pool rather than being mistakenly removed.

Query Sampling

After negative images are removed, CoMODAL selects query images from the remaining unlabeled pool. The query sampler combines two complementary signals: uncertain predictions and likely high-confidence false positives.

For uncertain predictions, CoMODAL estimates a high-uncertainty confidence interval from the detector’s score distribution and then applies diversity-based sampling using patch embeddings. This helps select samples that are both uncertain and visually diverse.

For high-confidence false positives, CoMODAL searches for confident predictions whose patch embeddings are dissimilar from previously annotated objects. These samples are important because confident false positives can strongly hurt detection performance.

Targeting High-Confidence False Positives

A detector can sometimes make confident but incorrect predictions. These errors are especially harmful because they appear early in the ranked prediction list and can significantly reduce Average Precision.

CoMODAL explicitly searches for such cases by comparing high-confidence prediction embeddings with embeddings from previously annotated objects. Predictions that are confident but far from known positives are treated as likely false positives and prioritized for annotation.

Accelerated Query Annotation

Once query images are selected, CoMODAL uses another committee of foundation models to generate candidate bounding boxes. The human annotator does not need to draw boxes from scratch; instead, they only approve or reject each proposed box.

This turns object detection annotation into a binary verification task. Accepted boxes are added to the labeled training set, and the detector is retrained for the next active learning iteration.

Training Loop

At each active learning iteration, the detector is trained using the mined negative images and all approved annotations collected so far. The updated detector is then used to produce new predictions for the next round of query sampling.

Through this loop, CoMODAL aims to improve detector performance while reducing both the number of annotated images and the effort required for each annotation.