arXiv:2610.24045 · v2
What Remains Without Leaking Test Labels? Exposing Evaluation Flaws in Graph-Level Anomaly Detection
摘要
Graph-level anomaly detection (GLAD) identifies unusual graphs in applications such as molecular screening, often using only normal graphs for training. Yet the evaluations of some detectors select models with test labels, and high benchmark scores alone do not show whether detectors recognize structural changes. We audit released code and compare the area under the receiver operating characteristic curve (AUROC) for model states (checkpoints) chosen with and without test labels from the same training run. We separately compare test-tuned hyperparameters with repository defaults and test sensitivity to structural changes while controlling graph size. We evaluate 21 detectors on the UB-GOLD benchmark and on synthetic and molecular graphs with controlled structural changes. About half of the checkpoint-selection gain is linked to choosing a checkpoint and reporting its performance on the same labeled test set. The mean gap persists across training seeds and when normal and anomalous graphs come from different datasets. Most detectors miss many tested structural changes that simple counts detect. With tuned hyperparameters, the two leaders remain the top two by AUROC under every checkpoint rule. Across nine main datasets, test-label checkpoint selection raises the mean AUROC of UB-GOLD's 12 detectors by 5.12 points relative to the final checkpoint.
作者备注：v2: Retitled. Adds a replication in UB-GOLD's Type III scenario, two extra training seeds, and three analyses of where the checkpoint-selection gain. Adds low-order controls to the structural tests and checks of test-time randomness and of the held-out split. Releases corrected Tox21 splits, per-graph scores and a supplementary study record. Text revised throughout. 56 pages, 9 figures