Adapting Generalist Vehicle Models for High-Speed MPC Across Terrains
Videos
Abstract
High-speed off-road autonomy requires precise closed-loop control for a target vehicle while remaining robust across changing terrains. Recent forward kinodynamic (FKD) prediction foundation models suggest a promising path, starting from a generalist model and specializing it to the target platform. However, effective specialization remains challenging, as it often requires substantial real-world data, and models adapted to one setting can still overfit to specific terrains or driving regimes.
We present OptCar (Optimized Car), a recipe for bridging the gap from generalist to specialist FKD models that preserves cross-terrain generalization while optimizing performance for a specific vehicle. OptCar introduces a history-conditioned dynamics adaptation module that encodes recent state-action observations into a dynamics context token, and then fine-tunes the generalist model using limited real-world data together with targeted synthetic rollouts from environment-specific system identification.
In closed-loop model predictive control (MPC) experiments across three terrains and an out-of-distribution cart-pulling task, the largest gains appear at 6 m/s, the highest speed evaluated and the regime in which slip dominates tracking error. On vegetation and dirt, the most slip-diverse terrain, OptCar reduces 6 m/s trajectory tracking error by roughly 55% relative to a fine-tuned AnyCar baseline, and remains the most accurate even when an unseen cart payload changes the dynamics. With only 5 minutes of real data per terrain, OptCar is competitive on road with a specialist trained on 30 minutes of road data, and substantially outperforms it once the terrain changes.
OptCar method
A bidirectional transformer compresses the recent state-action history into a single context vector that represents the active vehicle dynamics. This context conditions every block of the causal FKD rollout decoder through FiLM. Given a candidate action sequence, the decoder predicts future body-frame state increments for MPC without online optimization or an explicit terrain classifier.
OptCar begins with large-scale simulation pretraining, then specializes to the target vehicle using approximately five minutes of real driving per terrain. Each terrain log also calibrates a dynamic bicycle model that generates targeted synthetic rollouts. Real observations anchor the model to the physical platform, while synthetic trajectories expand coverage of the aggressive, high-slip regimes that are difficult to collect safely.
OptCar in closed loop
Experimental platform
