← 研踪 PaperTrailarXiv:2610.28814 · v2
T-WLA: Bridging Slow Vision and Fast Touch with a World-Language-Action Model
首次提交 2026-10-21 02:36
版本更新 2026-11-18 07:34
雷达观测 2026-11-19 19:08
摘要
In contact-rich manipulation, contact conditions can change before a visual policy updates its action plan. Recomputing the plan through the visual policy adds latency, while tactile feedback alone does not specify how the ongoing motion should be revised for the task. We introduce T-WLA, a world-language-action architecture that uses shared visual memory to guide both action generation and tactile correction. At each visual update, the visual branch integrates the current image, task instruction, and retrieved history into a visual memory that conditions base-action generation. The same memory is cached for a lightweight tactile editor, which combines it with the active action chunk and incoming touch to predict residual corrections to unexecuted actions without rerunning the visual branch. Future-image and future-pressure prediction provide training-only supervision, encouraging the visual memory and tactile features to capture interaction dynamics relevant to action generation and correction. We evaluate T-WLA on tactile-conditioned hand-motion prediction using EgoTouch and Tachin, and on four real-world dexterous manipulation tasks. On EgoTouch, T-WLA reduces finger-joint MPJPE by 30% relative to the strongest evaluated baseline. Across four real-robot tasks, T-WLA achieves 51.0% mean success versus 23.0% for the baseline, a gain of 28.0 percentage points. Videos and code are available at https://t-wla.github.io/.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-18 07:34 · 最新观测版本
T-WLA: Bridging Slow Vision and Fast Touch with a World-Language-Action Model
In contact-rich manipulation, contact conditions can change before a visual policy updates its action plan. Recomputing the plan through the visual policy adds latency, while tactile feedback alone does not specify how the ongoing motion should be revised for the task. We introduce T-WLA, a world-language-action architecture that uses shared visual memory to guide both action generation and tactile correction. At each visual update, the visual branch integrates the current image, task instruction, and retrieved history into a visual memory that conditions base-action generation. The same memory is cached for a lightweight tactile editor, which combines it with the active action chunk and incoming touch to predict residual corrections to unexecuted actions without rerunning the visual branch. Future-image and future-pressure prediction provide training-only supervision, encouraging the visual memory and tactile features to capture interaction dynamics relevant to action generation and correction. We evaluate T-WLA on tactile-conditioned hand-motion prediction using EgoTouch and Tachin, and on four real-world dexterous manipulation tasks. On EgoTouch, T-WLA reduces finger-joint MPJPE by 30% relative to the strongest evaluated baseline. Across four real-robot tasks, T-WLA achieves 51.0% mean success versus 23.0% for the baseline, a gain of 28.0 percentage points. Videos and code are available at https://t-wla.github.io/.
查看此版本解读 →查看此版本原文 ↗