Neural networks applied to financial time series operate in a regime of underspecification, where predictors achieve comparable out-of-sample error. Using large-scale volatility forecasting for S&P 500 stocks--twelve architecture-optimizer configurations, thirteen seeds each, evaluated on 564,795 held-out observations--we show that comparable test loss conceals measurably different predictors. In eleven of twelve comparisons, models trained from identical initialization and identical data order diverge $1.3$ to $7.8$ times further than repeated runs of either optimizer alone, ruling out initialization noise as the explanation. On a common scale, predictions differ by 5--17\% of the typical forecast error while the models' error magnitudes differ by less than 0.7\% of it. The divergence reshapes nonlinear response profiles, temporal dependence, and ultimately decisions: volatility-ranked portfolios trace a Sharpe-turnover frontier with up to 58\% turnover dispersion at comparable gross Sharpe ratios. In underspecified settings, optimization is a consequential source of inductive bias, and model evaluation must extend beyond scalar loss to functional and decision-level behavior.
Emanuele Rimoldi, Riccardo Neumarker, Tomaso Poggio, Daniel Mitropolsky
Can a model learn by trying to fool another model? We show that it can. Inspired by the Turing Test and its recent generalization to arbitrary distinguishers, we introduce a domain-based teacher-Turing-Test, in which a stronger teacher model serves as both the target of imitation and the distinguisher, and a weaker student model is trained to make the teacher attribute responses to itself. We use this self-attribution (or fooling) probability as a reinforcement learning reward, without ever providing the student ground-truth answers, nor requiring the teacher to generate responses. We test our approach independently in four domains with verifiable answers: CommonsenseQA (reasoning), GSM8K (mathematics), KodCode (programming), and Spider (SQL), training a Qwen2.5-1.5B student/imitator model using DeepSeek V4 Flash as teacher/distinguisher. Beyond improving at passing the Turing Test, the student improves on independent, held-out evaluations in all domains and runs. To probe why being mistaken for the teacher can improve capability, our analyses reveal that self-attribution is associated with correctness -- substantially more than formatting changes -- and that it rewards partial correctness. Our results demonstrate a new, simple teacher-student training paradigm with a private and/or high-cost teacher. They also suggest that indistinguishability need not merely measure behavior -- it can also be a mechanism for learning.