FrontierPhysics: Evaluating Agents for End-to-End Physics Research
Bingran You, Wenjun Ke, Ria Rosenauer, Bo Liu, Wenbo Chen, Min Chen, Shuqi Xu, Yue Ma 等 72 位作者
AI 速览：作者提出包含56个由物理学家基于自身研究设计的端到端物理研究基准，用确定性验证器与专家评分量表同时考察智能体研究的正确性与可发表性。
AI agents are playing an increasingly important role in physics research. However, existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. We introduce FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics. Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. Manual review of agent trajectories shows that frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors.
Xing Liu, Yifan Guo, Hongtai Lv, Lei Lou, Yuyi Tan, Jiang Du, Xijia Zhang, Ming Zhang 等 21 位作者
Intraoperative frozen-section diagnosis of central nervous system (CNS) lesions guides surgical decisions but is challenged by artefacts, diagnostic complexity and time constraints. Most pathology foundation models have limited exposure to CNS frozen tissue. Here we present FOCUS, a subspecialty foundation model pretrained on 27,930 frozen-section whole-slide images enriched for CNS tissue. FOCUS ranked first across all 11 diagnostic and 12 biomarker-prediction tasks in the internal test cohort, achieving mean AUCs of 0.969 and 0.877, respectively. This advantage was retained in the temporal and external test cohorts. We further organized a WHO-aligned diagnostic hierarchy, where FOCUS also outperformed comparator models. In a reader study, FOCUS assistance enabled junior readers to approach or exceed unassisted senior readers in hierarchical diagnostic accuracy and granularity. More importantly, FOCUS assistance enabled readers to correctly identify a mean of 63.0% of initially missed lymphoma cases (73.2% among junior readers), potentially avoiding unnecessary extensive resection. These findings highlight the potential of FOCUS to provide clinically actionable support for intraoperative CNS pathology and inform surgical decisions.