跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.23153 · v1

VIVID: Beyond Static Multi-Image Reasoning with Localized Visual Differences and Textual Entity Dynamics in Video MLLMs

VIVID:超越静态多图推理——视频多模态大语言模型中的局部视觉差异与文本实体动态

首次提交 2026-11-18 02:05
版本更新 2026-11-18 02:05
雷达观测 2026-11-18 12:34
arXiv ↗PDF ↗

摘要

Video question answering under tight frame budgets requires preserving localized motion and event order, yet current video multimodal large language models (MLLMs) reduce comprehension to static multi-image reasoning by discarding unselected frames and encoding keyframes independently: (1) Spatially, whole-frame pooling dilutes small moving targets into static backgrounds; (2) Temporally, dropping intermediate frames or averaging tokens erases inter-frame motion, confusing direction, speed, and order. We propose VIVID (Visual Inter-frame Variations and Individual Dynamics), which moves beyond static snapshots like human vision: keyframes capture core states while inter-frame Patch Diffs preserve neighboring changes within KfK_f keyframes. Autonomous Delta-Driven Keyframe Planning (ADKP) selects anchors from a 3 fps3\text{ fps} pool and feeds surrounding patch differences into Dual-Road Change Memory (DRCM), forming Dynamic Tokens at zero extra anchor tokens alongside a camera-stabilized Active Motion Region of Interest (ROI) and chronological Entity State Chains. Adding 0.22%0.22\% trainable parameters to a frozen 27B27\text{B} backbone, VIVID-Small (5050 examples, ∼1.5\sim 1.5 GPU hours) beats all eleven baselines in overall accuracy across 8,0838,083 questions and trails only VIVID-Large (3,6003,600 examples), which leads on MotionBench, Video-MME, and TempCompass.

作者备注:14 pages, 2 figures, 5 tables. Code: https://github.com/step-out/VIVID

AI 论文解读

基于摘要 · 非全文审阅

VIVID 通过保留关键帧间的局部视觉差异和实体状态链,让视频多模态大语言模型摆脱静态多图推理的局限。

正在读取账号与研究记录…

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-18 02:05 · 最新观测版本

VIVID: Beyond Static Multi-Image Reasoning with Localized Visual Differences and Textual Entity Dynamics in Video MLLMs

Video question answering under tight frame budgets requires preserving localized motion and event order, yet current video multimodal large language models (MLLMs) reduce comprehension to static multi-image reasoning by discarding unselected frames and encoding keyframes independently: (1) Spatially, whole-frame pooling dilutes small moving targets into static backgrounds; (2) Temporally, dropping intermediate frames or averaging tokens erases inter-frame motion, confusing direction, speed, and order. We propose VIVID (Visual Inter-frame Variations and Individual Dynamics), which moves beyond static snapshots like human vision: keyframes capture core states while inter-frame Patch Diffs preserve neighboring changes within KfK_f keyframes. Autonomous Delta-Driven Keyframe Planning (ADKP) selects anchors from a 3 fps3\text{ fps} pool and feeds surrounding patch differences into Dual-Road Change Memory (DRCM), forming Dynamic Tokens at zero extra anchor tokens alongside a camera-stabilized Active Motion Region of Interest (ROI) and chronological Entity State Chains. Adding 0.22%0.22\% trainable parameters to a frozen 27B27\text{B} backbone, VIVID-Small (5050 examples, ∼1.5\sim 1.5 GPU hours) beats all eleven baselines in overall accuracy across 8,0838,083 questions and trails only VIVID-Large (3,6003,600 examples), which leads on MotionBench, Video-MME, and TempCompass.

查看此版本解读 →查看此版本原文 ↗