← 研踪 PaperTrailarXiv:2608.29583 · v2
Drive the Thoughts: Runtime Monitoring of VLA Reasoning-Trajectory Consistency
首次提交 2026-08-30 14:03
版本更新 2026-11-13 14:14
雷达观测 2026-11-17 04:57
摘要
While machine learning models help autonomous vehicles (AVs) navigate complex environments, their black-box nature complicates validation and verification (V&V). Vision-language-action (VLA) models offer a unique opportunity: alongside trajectories, they generate an explicit chain-of-thought (CoT) explaining their decisions. We posit that this CoT can serve as a candidate specification to cross-check trajectories and expose unsafe behavior. To assess this, we curate DriveAlignBench, a dataset of 150 CoT-trajectory pairs from NVIDIA's Alpamayo 1.5 VLA, manually annotated for CoT reliability, CoT-trajectory consistency, and trajectory safety. We find that 33.3% of CoTs are unreliable, and among reliable CoTs, 26% have inconsistent trajectories. To detect these cases, we propose integrating a CoT-trajectory consistency check into a runtime monitor. This check is nontrivial because CoTs express open-vocabulary, scene-relative maneuvers, while trajectories are low-level ego-motion sequences whose semantics depend on road geometry and motion context. To bridge this gap, we develop a family of automated consistency monitors. The strongest, the lane-aware monitor, adds lane-relative evidence to the input of the judge model scoring consistency. This raises F1 across all four evaluated judges from 0.56-0.62 with raw waypoints to 0.65-0.82, showing that evidence representation improves detection across judges. Moreover, for reliable CoTs, inconsistency is associated with unsafe behavior: 27% of human-labeled inconsistent trajectories are unsafe compared to 3% of consistent ones, and the lane-aware monitor flags most of these unsafe cases. We release DriveAlignBench, the monitor implementations, and per-run results at https://github.com/less-lab-uva/drive-the-thoughts.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-13 14:14 · 最新观测版本
Drive the Thoughts: Runtime Monitoring of VLA Reasoning-Trajectory Consistency
While machine learning models help autonomous vehicles (AVs) navigate complex environments, their black-box nature complicates validation and verification (V&V). Vision-language-action (VLA) models offer a unique opportunity: alongside trajectories, they generate an explicit chain-of-thought (CoT) explaining their decisions. We posit that this CoT can serve as a candidate specification to cross-check trajectories and expose unsafe behavior. To assess this, we curate DriveAlignBench, a dataset of 150 CoT-trajectory pairs from NVIDIA's Alpamayo 1.5 VLA, manually annotated for CoT reliability, CoT-trajectory consistency, and trajectory safety. We find that 33.3% of CoTs are unreliable, and among reliable CoTs, 26% have inconsistent trajectories. To detect these cases, we propose integrating a CoT-trajectory consistency check into a runtime monitor. This check is nontrivial because CoTs express open-vocabulary, scene-relative maneuvers, while trajectories are low-level ego-motion sequences whose semantics depend on road geometry and motion context. To bridge this gap, we develop a family of automated consistency monitors. The strongest, the lane-aware monitor, adds lane-relative evidence to the input of the judge model scoring consistency. This raises F1 across all four evaluated judges from 0.56-0.62 with raw waypoints to 0.65-0.82, showing that evidence representation improves detection across judges. Moreover, for reliable CoTs, inconsistency is associated with unsafe behavior: 27% of human-labeled inconsistent trajectories are unsafe compared to 3% of consistent ones, and the lane-aware monitor flags most of these unsafe cases. We release DriveAlignBench, the monitor implementations, and per-run results at https://github.com/less-lab-uva/drive-the-thoughts.
查看此版本解读 →查看此版本原文 ↗