Automated research pipelines pair an LLM proposer with a verifier on an implicit bet: discovery quality rises with verification strength. Under a pre-registered protocol we test this bet in two real-data domains -- job-shop scheduling and vehicular edge offloading on Beijing T-Drive trajectories -- across 100 matched LLM/random pairs per domain and three verifier tiers. The manipulation binds -- a 429-case mutation battery shows scores rising across tiers -- yet the paired LLM-over-random advantage does not scale with it (A, one-sided p=0.1115; D, p=0.9994), and in D shrinks under the strict tier (margin contrast -0.040, p=0.0013). The two lower gates promote identical candidates in 100/100 and 99/100 pairs, so the strict order is untestable when they coincide. Constructively, LLM proposals carry a real paired advantage over random controls, but a post-hoc union-pool analysis shows the fastest path to better discovery is widening the pool, not tightening the gate: mixing the two pools adds +1.52 holdout-value units in domain A (one-sided Wilcoxon, n=100, p=0.0002) -- evidence that pool composition, not verification strength, is the lever. Criteria, ledgers and scripts are archived on Zenodo (DOI: 10.5281/zenodo.23308866).
Pre-registration is increasingly proposed as a trust mechanism for machine-learning research involving LLM-generated content, but its canonical form assumes a frozen protocol can be executed unchanged. Software-intensive studies violate this assumption: the protocol is partly implemented, and implementations carry defects. We report a three-case study of protocol repairs ("patches") from a program where an LLM proposes executable rules that a programmatic verifier gates, across production scheduling, incentive design, and task assignment. Each protocol required exactly one defect-repair patch: three events in total. None modified a numbered judgment criterion, its threshold, or the scoring rule, and none altered already collected data; two changed pre-registered design parameters before any data were collected, each with a recorded rationale. We propose a two-axis taxonomy of patches (defect class: environmental / feasibility / correctness, x timing: pre-data / post-data), introduce a verifiable non-impact claim for post-data repairs, and distill a nine-item checklist; all nine items were exercised by at least one patch event -- an observation of use, not a validation of sufficiency. The credibility of verifier-bound, LLM-proposer studies depends less on whether patches occur than on whether they are auditable.
Giovanni Zingaro, Yuan Wang, Deepak Meena, Peter Cheng, Russell Buchanan, John Quilty, Robert Gracie
The Jet Boring System (JBS) is a proprietary technology developed by Cameco Corporation to extract high-grade uranium ore at the Cigar Lake Mine in Northern Saskatchewan, Canada. The JBS remotely excavates ore using a high-pressure water jet that cuts cavities within an artificially frozen ore body. This study outlines a Deep Learning (DL)-based workflow that formulates the cavity prediction problem as a volumetric binary semantic segmentation task. A preprocessing pipeline assembles drilling and jetting records, cavity scans, and a mine-wide geology block model onto a common voxel grid, where each voxel in the prediction is labelled as consumed or non-consumed during jetting. A three-dimensional U-Net with a backfill residual branch is trained on historical field cavities. The model is benchmarked against a baseline consisting of a cylindrical cavity centered along the pilot hole, with a diameter defined by domain experts. The proposed workflow achieves an IoU of 0.601 $\pm$ 0.088 and an F$_1$-Score of 0.747 $\pm$ 0.077 on a held-out test set, improving on the cylindrical baseline currently used in cavity planning by 22.4% and 14.7%, respectively. An eXplainable Artificial Intelligence (XAI) feature attribution approach is applied to identify the most influential input driving the predicted cavity geometry. The accuracy of the proposed model demonstrates its potential to support mine planning.