Humanity’s List

AIarXiv

Ranked by formulaVerdict: Notable

StreamEval: Evaluating Interactive Streaming Video Generation Models

As streaming video generation advances, enabling real-time interaction with generated video is becoming increasingly important.

By StreamEval Team

Score██████░░░░5.6

Caveats

  • Preprint; not yet peer reviewed.

VerdictWorth a reader's time today.

Read the originalPDFCode

Abstract

As streaming video generation advances, enabling real-time interaction with generated video is becoming increasingly important. We introduce StreamEval, a unified benchmark for interactive streaming video generation spanning five tasks: avatar dialogue, avatar performance, world generation, instruction editing, and reference editing. StreamEval comprises 300 scripted test cases lasting 15-300s, each specifying initial assets, incrementally arriving multimodal requests, and explicit requirements for state changes and content preservation. Performance is assessed along four evaluation dimensions: task execution, state consistency, visual fidelity, and audiovisual coordination, instantiated through ten task-specific metrics on a 0-100 scale. Evaluating 37 distinct models across 300 test cases, we find that high visual quality and identity retention do not necessarily coincide with reliable request execution or sustained state consistency. In particular, request execution remains challenging: the highest task-specific scores are 36.90 for Following among seven avatar-performance models, 25.93 for Following in text-conditioned world generation, and 15.00 for Edit Success among three reported reference-editing models. These findings identify reliable request execution under timing and preservation constraints as a central challenge for advancing omni-modal real-time interactive systems. The leaderboard is available at https://streameval.ai

StreamEval Team

The editor's rubric

Heuristic review

Leverage4 / 5
A general-purpose tool used across several fields (Adam, ResNet, LoRA, next-generation sequencing).
24% of this desk’s rubric
Magnitude2 / 5
Solid incremental gain on a meaningful problem.
18% of this desk’s rubric
Evidence3 / 5
Solid: multiple benchmarks or cohorts, ablations, fair baselines, released code or data.
14% of this desk’s rubric
Novelty2 / 5
A new combination of known ideas.
16% of this desk’s rubric
Trajectory3 / 5
A clear path to scale.
18% of this desk’s rubric
Stakes2 / 5
Benefits a professional community (practitioners, clinicians, engineers).
10% of this desk’s rubric

Editor’s rationale

Heuristic triage from title and abstract text only, not a reading of the paper. Cues found: method (we propose, we report); breadth (many tasks); verification (multiple benchmarks); scale (scalable).

How the score was computed

rank-2026-10-18

Score██████░░░░5.6

Score = 10 × (65% × adjusted merit / 10 + 25% × attention + 10% × freshness)

Desks place stronger verdicts first, then use this score to order work within the same verdict.

Merit
5.6 / 10
Weighted rubric, evidence-gated.
Adjusted merit
4.6 / 10
Shrunk toward the desk prior by editor confidence (40%).
Attention
94%
Citations, upvotes, points, mentions.
Freshness
26%
Half-life decay since publication.
  • Citations0 (reference 15, via semantic-scholar, Oct 23, 2026, 13:24 UTC)
  • Influential citations0 (reference 3, via semantic-scholar, Oct 23, 2026, 13:24 UTC)
  • Hugging Face upvotes438 (reference 25, via hf-daily, Oct 24, 2026, 13:49 UTC)
  • GitHub stars242 (reference 250, via hf-daily, Oct 24, 2026, 13:49 UTC)

The record

  • Reviewed by heuristic-v10 on Oct 25, 2026, 13:49 UTC. Paper type: method.
  • Categories: cs.CV
  • TOP, No.1 in the Artificial Intelligence edition of October 25, 2026.