跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2609.28782 · v2

When a better baseline makes delta learning worse: separating residual scale from learnability to guide target design

首次提交 2026-09-24 04:57
版本更新 2026-11-17 01:01
雷达观测 2026-11-17 16:10
arXiv ↗PDF ↗

摘要

In scientific machine learning, ΔΔ-learning trains models on residual errors relative to approximate baselines, assuming that smaller residual scales inherently yield better downstream accuracy. Here, we demonstrate that residual scale alone is an insufficient heuristic for learnability. For molecular total energies, replacing a linear baseline with a neural network fitted to the same elemental counts reduced the residual scale, yet more than doubled GemNet's prediction error. Across local structural descriptors, more flexible baselines produced smaller residuals that varied sharply between structurally similar molecules. By contrast, semi-empirical baselines left larger residuals, yet yielded lower test error and improved out-of-domain transferability. Across the targets studied, scale-normalized graph Dirichlet roughness tracked scale-normalized prediction error, suggesting its use as a pre-training diagnostic for target selection. These results motivate baseline complementarity as a target-design principle in scientific machine learning: a useful baseline captures contributions the downstream architecture struggles to predict.

作者备注:16 pages, 4 main-text figures, and 22 pages of Supplementary Information provided as an ancillary PDF. Revised title and exposition, expanded Methods and Supplementary Information, and updated references

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-17 01:01 · 最新观测版本

When a better baseline makes delta learning worse: separating residual scale from learnability to guide target design

In scientific machine learning, ΔΔ-learning trains models on residual errors relative to approximate baselines, assuming that smaller residual scales inherently yield better downstream accuracy. Here, we demonstrate that residual scale alone is an insufficient heuristic for learnability. For molecular total energies, replacing a linear baseline with a neural network fitted to the same elemental counts reduced the residual scale, yet more than doubled GemNet's prediction error. Across local structural descriptors, more flexible baselines produced smaller residuals that varied sharply between structurally similar molecules. By contrast, semi-empirical baselines left larger residuals, yet yielded lower test error and improved out-of-domain transferability. Across the targets studied, scale-normalized graph Dirichlet roughness tracked scale-normalized prediction error, suggesting its use as a pre-training diagnostic for target selection. These results motivate baseline complementarity as a target-design principle in scientific machine learning: a useful baseline captures contributions the downstream architecture struggles to predict.

查看此版本解读 →查看此版本原文 ↗