← 研踪 PaperTrailarXiv:2605.31557 · v3
EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Vision
首次提交 2026-05-30 01:20
版本更新 2026-11-06 19:54
雷达观测 2026-11-09 11:19
摘要
Continuous episodic memory is a core capability for autonomous agents operating in dynamic, real-world environments, yet current streaming video benchmarks provide limited tools for diagnosing what models remember and for how long. We introduce EGOSTREAM, a diagnostic benchmark for streaming episodic memory evaluation in egocentric vision. EGOSTREAM organizes 2,250 curated questions along seven cognitive dimensions: detail, spatial, temporal, event, social, causal, and prospective memory. We introduce the Answer Validity Window (AVW), which specifies the temporal span an answer remains valid as the observed scene evolves. This allows us to expand the questions into 8,528 recall-conditioned evaluations, enabling controlled testing from instant to ultra-long-term recall while separating genuine model forgetting from natural world-state changes. We rigorously establish baseline performance through a unified streaming MLLM framework that compares several state-of-the-art memory-management mechanisms, covering sliding windows, attention sinks, KV-cache pruning, merging, adaptive head budgets, and offloading. Experiments on a unified Qwen3-VL backbone, with consistent trends on Molmo2-8B and InternVL3.5-8B, reveal that comparable aggregate accuracies mask starkly different memory profiles. For instance, token pruning preserves fine-grained details and temporal structure significantly better than token merging, adaptive head budgets yield the highest aggregate semantic accuracy, while quantized offloading rescues ultra-long-term recall. Ultimately, all in-cache memory policies operate well below real-time (>1s per frame), and top performing in-cache methods ceil at about 45% accuracy under a 9,804-token budget, exposing critical gaps in current architectures. EGOSTREAM provides the diagnostic testbed needed to close these gaps. Project website: https://coai-iplab.github.io/EgoStream/
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v3 · 2026-11-06 19:54 · 最新观测版本
EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Vision
Continuous episodic memory is a core capability for autonomous agents operating in dynamic, real-world environments, yet current streaming video benchmarks provide limited tools for diagnosing what models remember and for how long. We introduce EGOSTREAM, a diagnostic benchmark for streaming episodic memory evaluation in egocentric vision. EGOSTREAM organizes 2,250 curated questions along seven cognitive dimensions: detail, spatial, temporal, event, social, causal, and prospective memory. We introduce the Answer Validity Window (AVW), which specifies the temporal span an answer remains valid as the observed scene evolves. This allows us to expand the questions into 8,528 recall-conditioned evaluations, enabling controlled testing from instant to ultra-long-term recall while separating genuine model forgetting from natural world-state changes. We rigorously establish baseline performance through a unified streaming MLLM framework that compares several state-of-the-art memory-management mechanisms, covering sliding windows, attention sinks, KV-cache pruning, merging, adaptive head budgets, and offloading. Experiments on a unified Qwen3-VL backbone, with consistent trends on Molmo2-8B and InternVL3.5-8B, reveal that comparable aggregate accuracies mask starkly different memory profiles. For instance, token pruning preserves fine-grained details and temporal structure significantly better than token merging, adaptive head budgets yield the highest aggregate semantic accuracy, while quantized offloading rescues ultra-long-term recall. Ultimately, all in-cache memory policies operate well below real-time (>1s per frame), and top performing in-cache methods ceil at about 45% accuracy under a 9,804-token budget, exposing critical gaps in current architectures. EGOSTREAM provides the diagnostic testbed needed to close these gaps. Project website: https://coai-iplab.github.io/EgoStream/
查看此版本解读 →查看此版本原文 ↗