We build on 3DFA (Gkanatsios et al., 2025) as our base 3D policy and introduce , a learned orthogonal correction matrix that adapts the 3D RoPE positional encodings to account for unknown camera–robot extrinsics. Under fixed per-lab miscalibration — the realistic setting where each data source carries a consistent but unknown extrinsic error — our formulation recovers to near the performance of a model trained with ground-truth calibration, substantially outperforming a miscalibrated 3DFA without correction and a 2D baseline with no 3D coordinates. This confirms that the model can implicitly identify and compensate for a per-source calibration offset from the data alone. However, when miscalibration is randomized per episode, provides no benefit, suggesting the correction relies on consistent error structure across a source rather than frame-level adaptation. We further extend this approach to GR00T N1.5 by injecting 3D RoPE into its cross-attention layers and finetuning on a 1.4K-rollout subset of DROID. In this setting, 2D, 3D, and 3D+ variants converge to similar training loss, pointing to the need for larger-scale or more calibration-diverse data for the 3D signal to provide a clear advantage over the pretrained 2D backbone.
Conclusion
We presented a method for training 3D Vision-Language-Action models on large-scale robot datasets with unknown or noisy camera-robot extrinsics. Our key contribution is , a learned orthogonal correction to 3D RoPE positional encodings that is predicted from image content alone and enforced to lie in by construction via the matrix exponential. Under fixed per-source miscalibration, recovers most of the performance gap relative to a model with ground-truth calibration, without any explicit supervision on the extrinsics. This opens a practical path toward training large 3D-VLAs on datasets like DROID and OpenX, where calibration data is sparse, inconsistent, or entirely absent. A key limitation is that relies on consistent error structure within a source — randomly varying miscalibration remains unsolved, and likely requires a fundamentally different formulation, such as per-frame calibration estimation from multi-view geometric cues. We leave this as an important direction for future work.
