Publication
SmallDrive: An Efficient Vision-Language-Action Model for Autonomous Driving via Flow Matching
Atanas Poibrenski; Farzad Nozarian; Vahdat Abdelzad; Matthias Klusch; Christian Müller; Philipp Slusallek
In: GCPR 2026. German Conference on Pattern Recognition (GCPR-2026), September 22-25, Siegen, Germany, Springer, 2026.
Abstract
The integration of Vision-Language-Action (VLA) models into autonomous driving architectures promises richer scene understanding and auxiliary natural-language outputs, but typically imposes significant computational latency, rendering them ill-suited for high-frequency planning. In this paper, we propose SmallDrive, a VLA model designed to condition fast future trajectory generation on compact multimodal representations. Our approach relies on two core contributions. First, we propose a transformer-based flow matching trajectory head conditioned on the output features of a lightweight language model. We decouple trajectory planning from autoregressive textual output, enabling the synthesis of precise trajectories while retaining the ability to generate auxiliary textual descriptions on demand. Second, building on a compact query-based visual bottleneck, we introduce object-centric tokens that inject instance-level geometry and confidence from the detection decoder into the language model, strengthening its grounding to improve textual outputs and reduce trajectory–object collisions at negligible additional latency. Experiments on the nuScenes dataset demonstrate that SmallDrive achieves planning performance on par with state-of-the-art open-loop methods while excelling in complex turning scenarios, and our flow-matching formulation improves end-to-end latency by 8.5× over its autoregressive counterpart.