TwinPast: Benchmarking Historical Evidence Use in Streaming Video Understanding with Evidence Contrastive Video Pairs
Haotian Yang, Yong Chen, Cheng Peng, Haobo Wang, Zhongle Xie, Xuanru Li, Gang Chen
原文摘要
High accuracy on streaming video benchmarks does not show that a model uses visual history. Auditing OVO-Bench and StreamingBench, we find that recent frames alone suffice for 32% of sampled historical-task questions, and that the question and options alone also yield many correct answers. We introduce TwinPast, a benchmark of 350 human-reviewed video pairs spanning five historical tasks. The two videos in a pair share the recent video context, question, and ordered options, and differ in earlier events and therefore in the correct answer. Each video is evaluated independently and PairAcc requires both answers to be correct, so an answer determined by the shared inputs cannot solve a pair. Masking the annotated critical evidence lowers PairAcc from as high as 91% to at most 19%, whereas equal-area masking elsewhere changes it by at most 3.4 points. The strongest streaming system reaches 49% PairAcc, while general-purpose MLLMs given each complete video reach 67-91%, leaving substantial headroom. Streaming systems fail mainly by giving both histories the same answer. They give different valid answers on at most 57% of pairs, while the general-purpose MLLMs do so on 79-94%. Case studies further show wrong answers when the deciding evidence is still available at the final query.
cs.CV