← 研踪 PaperTrailarXiv:2611.25649 · v1
KeyMem: Temporal Occupancy Control for Robust Streaming VideoLLM Memory
首次提交 2026-11-19 15:32
版本更新 2026-11-19 15:32
雷达观测 2026-11-20 19:58
摘要
Streaming VideoLLMs retain a bounded cache of visual tokens. Yet score-based eviction lets the stream determine its own temporal occupancy, so a short salient segment can monopolize the cache and displace evidence needed later. We identify this uncontrolled temporal occupancy as a fundamental limitation and introduce KeyMem, a training-free wrapper that rank-normalizes any base score, divides the video stream into fixed-duration temporal buckets and limits how many visual tokens each bucket can retain, and optionally samples survivors with a secret key. This hard cap bounds a segment's memory share from its first chunk. On Qwen2.5-VL-7B, 20 s of burned-in captions occupies 84% of an 8 s memory and lowers accuracy over all clips from 79.4% to 58.3%. Across five natural stressors, three VideoLLMs and two budgets, temporal quotas raise the worst accuracy of attention-based retention at the smaller budget from 42.2% to 73.3-76.1% and exceed that of InfiniPot-V by 6.1-16.7% on two of the three models. Deterministic and keyed quotas perform comparably under natural stress. Under an adaptive adversary, a secret key preserves 4.4-9.5% more early-clip accuracy than a leaked key, but does not consistently beat the best deterministic quota. A decide-once variant keeps old evidence that fresh draws erode. Temporal allocation is the primary robustness control, with secrecy reserved for adversarial predictability.
作者备注:23 pages, 4 figures
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-19 15:32 · 最新观测版本
KeyMem: Temporal Occupancy Control for Robust Streaming VideoLLM Memory
Streaming VideoLLMs retain a bounded cache of visual tokens. Yet score-based eviction lets the stream determine its own temporal occupancy, so a short salient segment can monopolize the cache and displace evidence needed later. We identify this uncontrolled temporal occupancy as a fundamental limitation and introduce KeyMem, a training-free wrapper that rank-normalizes any base score, divides the video stream into fixed-duration temporal buckets and limits how many visual tokens each bucket can retain, and optionally samples survivors with a secret key. This hard cap bounds a segment's memory share from its first chunk. On Qwen2.5-VL-7B, 20 s of burned-in captions occupies 84% of an 8 s memory and lowers accuracy over all clips from 79.4% to 58.3%. Across five natural stressors, three VideoLLMs and two budgets, temporal quotas raise the worst accuracy of attention-based retention at the smaller budget from 42.2% to 73.3-76.1% and exceed that of InfiniPot-V by 6.1-16.7% on two of the three models. Deterministic and keyed quotas perform comparably under natural stress. Under an adaptive adversary, a secret key preserves 4.4-9.5% more early-clip accuracy than a leaked key, but does not consistently beat the best deterministic quota. A decide-once variant keeps old evidence that fresh draws erode. Temporal allocation is the primary robustness control, with secrecy reserved for adversarial predictability.
查看此版本解读 →查看此版本原文 ↗