← 研踪 PaperTrailarXiv:2610.38171 · v1
B2P-Corr: Batch-to-Population Gradient Estimators for Non-Decomposable Correlation Losses
首次提交 2026-10-28 10:22
版本更新 2026-10-28 10:22
雷达观测 2026-10-30 09:53
摘要
Correlation metrics such as the Pearson Correlation Coefficient (PCC) and Concordance Correlation Coefficient (CCC) are standard regression metrics and learning objectives, but their non-decomposable nature poses a challenge for stochastic optimization. Since these metrics depend on dataset-level moments (mean, variance, covariance), naive mini-batch training with local statistics yields biased, high-variance gradients whose expectation does not align with the population gradient. To bridge this ``batch-to-population'' (B2P) gap, we propose \textbf{B2P-Corr}, a novel framework that transforms correlation objectives into decomposable pseudo-losses driven by lightweight global ``moment sketches''. We introduce two efficient variants: \textbf{B2P-EMA}, which utilizes exponential moving averages and stop-gradient operators to ensure asymptotic gradient tracking with $O(B)$ cost ($B$ is batch size); and \textbf{B2P-U}, which uses sampled cross-batch U-statistic estimators and a reservoir-based stabilization with an $O(B)$ implementation. Theoretically, we quantify the $O(B^{-1})$ bias of naive mini-batch optimization, prove a moving-target tracking result for B2P-EMA under a fast-sketch / slow-parameter regime, and provide an idealized control-variate analysis for B2P-U. Experiments on two synthetic and three real-world datasets demonstrate B2P-Corr achieves stable optimization and higher correlation than naive mini-batch optimization with negligible overhead.
作者备注：Accepted by NeurIPS 2026
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v1 · 2026-10-28 10:22 · 最新观测版本
B2P-Corr: Batch-to-Population Gradient Estimators for Non-Decomposable Correlation Losses
Correlation metrics such as the Pearson Correlation Coefficient (PCC) and Concordance Correlation Coefficient (CCC) are standard regression metrics and learning objectives, but their non-decomposable nature poses a challenge for stochastic optimization. Since these metrics depend on dataset-level moments (mean, variance, covariance), naive mini-batch training with local statistics yields biased, high-variance gradients whose expectation does not align with the population gradient. To bridge this ``batch-to-population'' (B2P) gap, we propose \textbf{B2P-Corr}, a novel framework that transforms correlation objectives into decomposable pseudo-losses driven by lightweight global ``moment sketches''. We introduce two efficient variants: \textbf{B2P-EMA}, which utilizes exponential moving averages and stop-gradient operators to ensure asymptotic gradient tracking with $O(B)$ cost ($B$ is batch size); and \textbf{B2P-U}, which uses sampled cross-batch U-statistic estimators and a reservoir-based stabilization with an $O(B)$ implementation. Theoretically, we quantify the $O(B^{-1})$ bias of naive mini-batch optimization, prove a moving-target tracking result for B2P-EMA under a fast-sketch / slow-parameter regime, and provide an idealized control-variate analysis for B2P-U. Experiments on two synthetic and three real-world datasets demonstrate B2P-Corr achieves stable optimization and higher correlation than naive mini-batch optimization with negligible overhead.
查看此版本解读 →查看此版本原文 ↗