{"id":174,"date":"2025-12-12T17:36:01","date_gmt":"2025-12-12T17:36:01","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2025team1\/?page_id=174"},"modified":"2025-12-12T18:39:01","modified_gmt":"2025-12-12T18:39:01","slug":"text-to-trajectory-model","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2025team1\/text-to-trajectory-model\/","title":{"rendered":"Text-to-Trajectory Model"},"content":{"rendered":"\n

We tested two different architectures to arrive at our final text-to-trajectory model – a diffusion transformer, and an autoregressive transformer.<\/p>\n\n\n\n

DIRECTOR<\/h2>\n\n\n\n
\n

The first model we tested is DIRECTOR [1], a diffusion-transformer based model that generates a full trajectory based on a text input that conditions the diffusion generation.

It is initially trained on The Exceptional Trajectories dataset, created by the same authors as this model. <\/p>\n\n\n\n

\"\"
Fig. 1 – DIRECTOR Model Architecture<\/figcaption><\/figure>\n<\/div>\n\n\n\n

GenDoP<\/h2>\n\n\n\n
\n

The second model we tested was GenDoP [2]. Opposed to the diffusion-based DIRECTOR, it is autoregressive, meaning a text encoder is the initial input, along with each previously generated trajectory token, and all of that is used to generate the next trajectory token.

It is initially trained using the GenDoP dataset, from the authors of the paper itself.<\/p>\n\n\n\n

\"\"
Fig. 2 – GenDoP Model Architecture<\/figcaption><\/figure>\n<\/div>\n\n\n\n

Results Comparison<\/h2>\n\n\n\n

We trained these models further, pruning the datasets and fine-tuning them to achieve the best results. Below are some comparisons between the two.<\/p>\n\n\n\n

    \n
  1. “Camera tilts down”<\/li>\n<\/ol>\n\n\n\n
    \n