Changhao Li, Bradley A. Chase, Veena Vijayakumar, Mathys Rennela, Alessandro Cosentino, Naixu Guo, Sebastian Hassinger, Farrokh Labib 等 14 位作者
Quantum open-source software provides essential infrastructure for quantum science, and artificial intelligence (AI) is reshaping its capabilities and transforming its development. However, verifying quantum software at scale remains difficult, and AI-generated contributions can increase the workload of maintainers. In this Perspective, we examine verification as a central requirement for reliable AI-assisted quantum software development and research. We analyze public repositories, records covering 110 repositories from five annual quantum open-source hackathons that we organize, and responses from more than 1600 participants in a community survey. Our analysis connects limitations in automated verification with the review burden on quantum software maintainers. We propose verification practices and contribution standards that require contributors to support their changes with reproducible evidence for maintainer review. We also examine how quantum open-source software can provide infrastructure for AI-driven research whose results the community can independently verify and build upon. We maintain an open-source companion repository to support reproducibility and help projects adopt and refine these proposals. Together, these contributions help the quantum open-source software community translate advances in AI into reliable quantum research through reproducible verification.
Elad Ben Avraham Amazon, ChangHao Li Georgia Institute of Technology, Ron Dorfman Amazon, Roy Ganz, Oren Nuriel Amazon, Amir Dudai Amazon, Aviad Aberdam Amazon, Noah Flynn University of California, Berkeley and Amazon 等 11 位作者
Deep Research Agents generate analyst-grade reports, yet evaluating them remains challenging due to the absence of a single ground truth and the multidimensional nature of research quality. Recent benchmarks propose distinct methodologies, yet they suffer from the Mirage of Synthesis, where strong surface-level fluency and citation alignment can obscure underlying factual and reasoning defects. We characterize this gap by introducing a taxonomy across four verticals that exposes a critical capability mismatch: static evaluators inherently lack the tool-use capabilities required to assess temporal validity and factual correctness. To address this, we propose DREAM (Deep Research Evaluation with Agentic Metrics), a framework that instantiates the principle of capability parity by making evaluation itself agentic. DREAM structures assessment through an evaluation protocol combining query-agnostic metrics with adaptive metrics generated by a tool-calling agent, enabling temporally aware coverage, grounded verification, and systematic reasoning probes. Controlled evaluations demonstrate DREAM is significantly more sensitive to factual and temporal decay than existing benchmarks, offering a scalable, reference-free evaluation paradigm.
尚无 AI 解读，展示原文摘要
cs.CL · cs.AI
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表