arXiv:2509.23219 · v3

WirelessMathBench-XL: An Auditable Benchmark for Wireless Mathematical Reasoning

首次提交 2025-09-27 17:58

版本更新 2026-11-08 10:16

雷达观测 2026-11-10 12:16

摘要

Benchmarks built from arXiv papers can overlap the arXiv text used in LLM pretraining, yet technical-domain releases rarely ship the per-item evidence needed to check this. We introduce a reverse-probe n-gram audit that indexes the benchmark side and streams the corpus past it, so memory scales with the benchmark rather than the corpus and flags on the distributed problems can be inspected and recomputed. We instantiate it in WirelessMathBench-XL, a 4,027-problem benchmark for wireless mathematical reasoning built from 836 arXiv papers across 20 subfields, with multiple-choice, progressive fill-in, and full-equation-completion formats. Streaming 12.9 B RedPajama-arXiv 13-grams yields a strict zero-hit view $\mathcal{S}_0$ covering 95.7% of the problems, and shows that the Dolma and ProofPile-2 arXiv slices repackage the same RedPajama dump. Filtering to $\mathcal{S}_0$ moves every evaluated model by less than 1 pp. With 30/800 flagged test items, this comparison bounds detected-overlap inflation at 0.31-0.51 pp for frontier models rather than testing for contamination, and it does not cover paraphrase, target-answer, alternate-version, post-training, or closed-corpus exposure. Frontier models reach 86.5-91.3%, and GRPO on the released training split lifts a 7B model 25.1 pp above a size-matched general-math control. We release source-paper identifiers, per-problem audit and threshold metadata, license-aware data files, audit and bootstrap scripts, Croissant metadata, and a Datasheet.

作者备注：NeurIPS 2026 Evaluations and Datasets Track. Project page: https://lixin.ai/WirelessMathBench-XL/