RAP – 3D Rasterization Augmented End-to-End Planning
This work challenges the need for high-quality RGB rendering by proposing that planning task depends on geometry and dynamics rather than textures or lightning, enabling easier data augmentations.
- Cross-agent synthesis: Generates diverse realistic views and interactions from other agents perspective.
- Recovery-oriented perturbations: Increases model robustness by simulating recovery from distribution shifts.
- Latent space alignment: Uses MSE loss (spatial-level alignment) and BCE loss (global-level alignment) to enforce feature consistency between the rasterized and real inputs.
Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?
Existing datasets do not fully capture the complexity and variability of real world driving. InterPlan addresses this by introducing a benchmark to evaluate planning approaches on realistic long-tail scenarios. This work also proposes a hybrid planner combining LLM based behavioural planning with rule based motion planning.
- Augmented scenarios include jaywalking pedestrians, accident scenarios, overtaking maneuvers and construction scenarios.
- LLM based behavioural planner provides high level decision making with semantic understanding while rule based motion planner is used for precise trajectory generation.
SteerVLA: Steering Vision-Language-Action Models in Long-Tail Driving Scenarios
Proposes using natural language prompts to guide Vision-Language-Action models in handling complex, long-tail driving scenarios incorporating high-level semantic reasoning from VLMs into the lower level control loop.
- Automatic labelling pipeline uses LLMs to generate reasoning traces from driving data creating chain-of-thought explanations
- VLM provides high level semantic reasoning about the scene and VLA model execeytes precise control commands based on combined visual and semantic input.
References
[1] Feng, Lan, et al. “Rap: 3d rasterization augmented end-to-end planning.” arXiv preprint arXiv:2510.04333 (2025).
[2] Hallgarten, Marcel, et al. “Can vehicle motion planning generalize to realistic long-tail scenarios?.” 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024.
[3] Gao, Tian, et al. “SteerVLA: Steering Vision-Language-Action Models in Long-Tail Driving Scenarios.” arXiv preprint arXiv:2602.08440 (2026).