Graph-level anomaly detection (GLAD) identifies unusual graphs in applications such as molecular screening, often using only normal graphs for training. Yet the evaluations of some detectors select models with test labels, and high benchmark scores alone do not show whether detectors recognize structural changes. We audit released code and compare the area under the receiver operating characteristic curve (AUROC) for model states (checkpoints) chosen with and without test labels from the same training run. We separately compare test-tuned hyperparameters with repository defaults and test sensitivity to structural changes while controlling graph size. We evaluate 21 detectors on the UB-GOLD benchmark and on synthetic and molecular graphs with controlled structural changes. About half of the checkpoint-selection gain is linked to choosing a checkpoint and reporting its performance on the same labeled test set. The mean gap persists across training seeds and when normal and anomalous graphs come from different datasets. Most detectors miss many tested structural changes that simple counts detect. With tuned hyperparameters, the two leaders remain the top two by AUROC under every checkpoint rule. Across nine main datasets, test-label checkpoint selection raises the mean AUROC of UB-GOLD's 12 detectors by 5.12 points relative to the final checkpoint.
Jingsheng Zheng, Jintian Zhang, Yujie Luo, Yuren Mao, Yunjun Gao Zhejiang University, Lun Du Ant Group, Huajun Chen Zhejiang University and ZJU-Hangzhou Global Scientific and Technological Innovation Center, Ningyu Zhang Zhejiang University
Autonomous machine learning agents have revolutionized scientific discovery, yet they remain constrained by a Generate-Execute-Feedback paradigm. Previous approaches suffers from a severe Execution Bottleneck, as hypothesis evaluation relies strictly on expensive physical execution. To bypass these physical constraints, we internalize execution priors to substitute costly runtime checks with instantaneous predictive reasoning, drawing inspiration from World Models. In this work, we formalize the task of Data-centric Solution Preference and construct a comprehensive corpus of 18,438 pairwise comparisons. We demonstrate that LLMs exhibit significant predictive capabilities when primed with a Verified Data Analysis Report, achieving 61.5% accuracy and robust confidence calibration. Finally, we instantiate this framework in ForeAgent, an agent that employs a Predict-then-Verify loop, achieving a 6x acceleration in convergence while surpassing execution-based baselines by +6%.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表
Yujie Luo, Zhuoyun Yu, Xuehai Wang, Yuqi Zhu Zhejiang University, Ningyu Zhang Zhejiang University, Lanning Wei AntGroup, Lun Du Ant Group, Da Zheng Ant Group 等 9 位作者
Replicating AI research is a crucial yet challenging task for large language model (LLM) agents. Existing approaches often struggle to generate executable code, primarily due to insufficient background knowledge and the limitations of retrieval-augmented generation (RAG) methods, which fail to capture latent technical details hidden in referenced papers. Furthermore, previous approaches tend to overlook valuable implementation-level code signals and lack structured knowledge representations that support multi-granular retrieval and reuse. To overcome these challenges, we propose Executable Knowledge Graphs (xKG), a pluggable, paper-centric knowledge base that automatically integrates code snippets and technical insights extracted from scientific literature. When integrated into three agent frameworks with two different LLMs, xKG shows substantial performance gains (10.9% with o3-mini) on PaperBench, demonstrating its effectiveness as a general and extensible solution for automated AI research replication.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · 2026 · 正式发表