arXiv:2610.28402 · v3

StreamEval: Evaluating Interactive Streaming Video Generation Models

首次提交 2026-10-20 23:47

版本更新 2026-10-26 11:46

雷达观测 2026-10-29 00:36

摘要

We introduce StreamEval with four key features: (1) First interactive streaming video generation benchmark. (2) Fully open source. Benchmark data, evaluation code, protocols, and scoring rules are publicly available. (3) Comprehensive coverage. Five tasks cover avatar dialogue, avatar performance, world generation, instruction editing, and reference editing, with 300 scripted cases lasting 15 to 300 seconds. Ten metrics assess task execution, state consistency, visual fidelity, and audiovisual coordination. (4) Monthly updates. We will evaluate new and updated models and publish leaderboard results monthly. Results from 37 models show that high visual quality and identity retention do not necessarily coincide with reliable request execution or sustained state consistency. The leaderboard is available at https://streameval.ai.