← 研踪 PaperTrailarXiv:2611.23153 · v1
VIVID: Beyond Static Multi-Image Reasoning with Localized Visual Differences and Textual Entity Dynamics in Video MLLMs
VIVID:超越静态多图推理——视频多模态大语言模型中的局部视觉差异与文本实体动态
首次提交 2026-11-18 02:05
版本更新 2026-11-18 02:05
雷达观测 2026-11-18 12:34
摘要
Video question answering under tight frame budgets requires preserving localized motion and event order, yet current video multimodal large language models (MLLMs) reduce comprehension to static multi-image reasoning by discarding unselected frames and encoding keyframes independently: (1) Spatially, whole-frame pooling dilutes small moving targets into static backgrounds; (2) Temporally, dropping intermediate frames or averaging tokens erases inter-frame motion, confusing direction, speed, and order. We propose VIVID (Visual Inter-frame Variations and Individual Dynamics), which moves beyond static snapshots like human vision: keyframes capture core states while inter-frame Patch Diffs preserve neighboring changes within keyframes. Autonomous Delta-Driven Keyframe Planning (ADKP) selects anchors from a pool and feeds surrounding patch differences into Dual-Road Change Memory (DRCM), forming Dynamic Tokens at zero extra anchor tokens alongside a camera-stabilized Active Motion Region of Interest (ROI) and chronological Entity State Chains. Adding trainable parameters to a frozen backbone, VIVID-Small ( examples, GPU hours) beats all eleven baselines in overall accuracy across questions and trails only VIVID-Large ( examples), which leads on MotionBench, Video-MME, and TempCompass.
作者备注:14 pages, 2 figures, 5 tables. Code: https://github.com/step-out/VIVID
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-18 02:05 · 最新观测版本
VIVID: Beyond Static Multi-Image Reasoning with Localized Visual Differences and Textual Entity Dynamics in Video MLLMs
Video question answering under tight frame budgets requires preserving localized motion and event order, yet current video multimodal large language models (MLLMs) reduce comprehension to static multi-image reasoning by discarding unselected frames and encoding keyframes independently: (1) Spatially, whole-frame pooling dilutes small moving targets into static backgrounds; (2) Temporally, dropping intermediate frames or averaging tokens erases inter-frame motion, confusing direction, speed, and order. We propose VIVID (Visual Inter-frame Variations and Individual Dynamics), which moves beyond static snapshots like human vision: keyframes capture core states while inter-frame Patch Diffs preserve neighboring changes within keyframes. Autonomous Delta-Driven Keyframe Planning (ADKP) selects anchors from a pool and feeds surrounding patch differences into Dual-Road Change Memory (DRCM), forming Dynamic Tokens at zero extra anchor tokens alongside a camera-stabilized Active Motion Region of Interest (ROI) and chronological Entity State Chains. Adding trainable parameters to a frozen backbone, VIVID-Small ( examples, GPU hours) beats all eleven baselines in overall accuracy across questions and trails only VIVID-Large ( examples), which leads on MotionBench, Video-MME, and TempCompass.
查看此版本解读 →查看此版本原文 ↗