← 研踪 PaperTrailarXiv:2603.14938 · v2
FAR-Drive: Frame-AutoRegressive Video Generation for Autonomous Driving
首次提交 2026-03-16 15:40
版本更新 2026-11-17 16:36
雷达观测 2026-11-18 19:42
摘要
Despite rapid progress in autonomous driving, interactive simulation remains a fundamental bottleneck. Existing closed-loop simulators generate video in multi-frame chunks to maintain short-term stability, yet this coarse granularity weakens the fine-grained interaction between agent actions and visual dynamics. Directly adopting frame-level autoregressive generation restores this interaction but sharply intensifies two challenges: long-horizon consistency and autoregressive stability under iterative self-conditioning. In this paradigm, each step is a single action-conditioned update, which also imposes strict low-latency demands since every step requires a full inference pass. In this work, we propose FAR-Drive, a Frame-level AutoRegressive video generation framework for autonomous driving, which integrates a multi-view diffusion transformer with fine-grained structured control to produce geometrically consistent multi-camera videos. At the core of our framework, we design a two-stage training strategy: (i) adaptive reference-horizon conditioning, which expands the temporal conditioning window for long-horizon consistency, and (ii) blend-forcing training, which gradually shifts from ground-truth to self-conditioned inputs to improve autoregressive stability. We further integrate system-level inference optimizations to satisfy the low-latency constraint. Experiments on nuScenes show that our method achieves state-of-the-art results among autoregressive driving video generation methods while keeping sub-second latency on a single GPU.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-17 16:36 · 最新观测版本
FAR-Drive: Frame-AutoRegressive Video Generation for Autonomous Driving
Despite rapid progress in autonomous driving, interactive simulation remains a fundamental bottleneck. Existing closed-loop simulators generate video in multi-frame chunks to maintain short-term stability, yet this coarse granularity weakens the fine-grained interaction between agent actions and visual dynamics. Directly adopting frame-level autoregressive generation restores this interaction but sharply intensifies two challenges: long-horizon consistency and autoregressive stability under iterative self-conditioning. In this paradigm, each step is a single action-conditioned update, which also imposes strict low-latency demands since every step requires a full inference pass. In this work, we propose FAR-Drive, a Frame-level AutoRegressive video generation framework for autonomous driving, which integrates a multi-view diffusion transformer with fine-grained structured control to produce geometrically consistent multi-camera videos. At the core of our framework, we design a two-stage training strategy: (i) adaptive reference-horizon conditioning, which expands the temporal conditioning window for long-horizon consistency, and (ii) blend-forcing training, which gradually shifts from ground-truth to self-conditioned inputs to improve autoregressive stability. We further integrate system-level inference optimizations to satisfy the low-latency constraint. Experiments on nuScenes show that our method achieves state-of-the-art results among autoregressive driving video generation methods while keeping sub-second latency on a single GPU.
查看此版本解读 →查看此版本原文 ↗