← 研踪 PaperTrailCVF · CVPR 2026:cvf:8c37b2c9aa7cc98af018ef767a73d8b6 · 记录 #1
W2W: Language-Model-Based Trajectory Prediction with Reinforcement Learning
来源日期(仅到年份) 2026
来源记录日期 2026
雷达观测 2026-11-05 02:36
摘要
Pedestrian trajectory prediction is crucial for applications such as autonomous driving and social robots. Recently, language model (LM)-based trajectory prediction has offered both prediction accuracy and interpretability. However, the L2 loss commonly used in trajectory prediction cannot be directly applied to LM optimization, resulting in degraded prediction performance. Moreover, current LM-based trajectory prediction methods lack explicit expressions of social interactions, and their scene descriptions are overly simplistic, making it challenging to impose practical scene constraints. To address these issues, we propose Write-to-Walk (W2W). First, we convert observed trajectories and interaction cues (companion/following/obstacle) into parsable textual prompts, so that interaction semantics are expressed more explicitly in the model input. Afterward, a T5-Small backbone is trained in a two-stage manner: (1) Full-parameter supervised fine-tuning with cross-entropy loss for language learning, enabling formatted question answering; (2) Reinforcement Learning (RL) to optimize W2W, where a reward function combining ADE error and off-road penalties strengthens scene constraints, producing future trajectories consistent with the scene context and further improving prediction accuracy. Experiments on benchmark datasets (ETH/UCY and SDD) demonstrate that W2W remains competitive with recent LM-based prediction methods and strong trajectory prediction baselines on ADE/FDE. The interpretability of LMs further supports W2W's deployment in safety-critical domains such as autonomous driving.
论文发现出处
这些页面提供论文线索,不代表同行评审或质量认证。
观测到的版本
平台记录号代表本站观测到的元数据变化,不是作者修订号。来源日期仅到月份或年份时,按该月或该年首日存档。
记录 #1 · 2026 · 最新观测版本
W2W: Language-Model-Based Trajectory Prediction with Reinforcement Learning
Pedestrian trajectory prediction is crucial for applications such as autonomous driving and social robots. Recently, language model (LM)-based trajectory prediction has offered both prediction accuracy and interpretability. However, the L2 loss commonly used in trajectory prediction cannot be directly applied to LM optimization, resulting in degraded prediction performance. Moreover, current LM-based trajectory prediction methods lack explicit expressions of social interactions, and their scene descriptions are overly simplistic, making it challenging to impose practical scene constraints. To address these issues, we propose Write-to-Walk (W2W). First, we convert observed trajectories and interaction cues (companion/following/obstacle) into parsable textual prompts, so that interaction semantics are expressed more explicitly in the model input. Afterward, a T5-Small backbone is trained in a two-stage manner: (1) Full-parameter supervised fine-tuning with cross-entropy loss for language learning, enabling formatted question answering; (2) Reinforcement Learning (RL) to optimize W2W, where a reward function combining ADE error and off-road penalties strengthens scene constraints, producing future trajectories consistent with the scene context and further improving prediction accuracy. Experiments on benchmark datasets (ETH/UCY and SDD) demonstrate that W2W remains competitive with recent LM-based prediction methods and strong trajectory prediction baselines on ADE/FDE. The interpretability of LMs further supports W2W's deployment in safety-critical domains such as autonomous driving.
查看此版本解读 →查看此版本原文 ↗