跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+0新增论文
+0其中 AI 相关
+1已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-08 21:48 · 全站最近入库：2026-11-08 06:02各信源进度 →

统计口径与数据新鲜度

统计 2026-11-07 22:25 — 2026-11-08 22:25（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-08 21:48最近发现新增版本：2026-11-08 06:02库内论文最新更新时间：2026-11-06 02:59

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

17,100 已收录论文14,661 AI 相关4,057 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 460 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-08 21:46（北京时间）
版本更新 · v2arXiv:2610.36835

Joint Distributional Copula Regression for Insurance Claim Frequency and Severity via Boosting

Banghee So, Na Zhang

原文摘要

In this paper, we introduce a model-based boosting approach for fitting structured additive distributional copula regression models for bivariate responses, with particular emphasis on jointly modeling insurance claim frequency and severity. The approach uses regression-type base learners, such as linear terms and splines with low degrees of freedom. It builds on generalized additive models for location, scale, and shape (GAMLSS), which extend generalized additive models (GAMs) by allowing multiple distributional parameters, rather than only the conditional mean, to depend on covariates. The structured additive predictors make the model inherently interpretable, offering greater transparency and auditability than black-box approaches. These features are particularly relevant in regulated actuarial settings and applications requiring explainable artificial intelligence. By estimating the entire conditional joint distribution, the framework also enables insurance practitioners to examine conditional quantiles, tail behavior, and prediction intervals of risks. The copula component captures dependence between claim frequency and severity and allows practitioners to investigate how this dependence varies with policyholder characteristics, potentially improving predictive performance. Moreover, by incorporating modern machine-learning techniques, specifically boosting, the proposed model improves prediction accuracy while retaining interpretability. We assess the proposed framework using both simulated and real data. The results demonstrate its ability to recover the true covariate effects on the marginal distributional parameters and the dependence structure, select informative variables, and achieve predictive performance competitive with standard benchmark models. We also illustrate how to interpret the fitted model.

stat.ML · cs.LG · stat.ME

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38122

MiniMMD: Minimization of MMD for Sampling, Inference and Integration

Zonghao Chen

原文摘要

Many problems in computational statistics and machine learning can be formulated as approximating a target distribution by minimizing a discrepancy over a restricted class of candidate distributions. This minimum-discrepancy viewpoint appears, for example, in numerical integration, sampling, generative modelling, and statistical inference, but these instances are often studied separately in the literature. This thesis develops the first unified treatment of these many problems through the minimization of the maximum mean discrepancy (MMD), a kernel-based discrepancy between probability measures. The central theme is that MMD is not only a statistically meaningful distance between probability distributions, but also a tractable optimization objective to design practical and efficient algorithms. This thesis summarizes the recent theoretical and empirical advancements of such minimum-MMD based algorithms.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2604.15742

Collective Kernel EFT for Pre-activation ResNets

Hidetoshi Kawase, Toshihiro Ota

原文摘要

In finite-width deep neural networks, the empirical kernel $G$ evolves stochastically across layers. We develop a collective kernel effective field theory (EFT) for a simplified pre-activation residual model. Exact conditional Gaussianity of the residual increment gives finite-width discrete identities for $G$ and its sigma-kernel source. A projected $G$-only closure and a continuous-depth approximation define reference equations for the mean kernel $K_0$, the kernel covariance $V_4$, and the next-to-leading mean correction $K_1$. Numerical tests show that this closure captures the leading mean-kernel evolution, whereas its covariance and next-to-leading accuracy depend on sigma-kernel modes omitted from the $G$-only state. For the scalar ($N=1$) model with $\tanh$ activation, a Hermite--Galerkin extension agrees with the extrapolated $K_1$ value and reduces the $V_4$ endpoint error by $85$--$93\%$ relative to the $G$-only reference. These results clarify the range of validity of the equal-time Markov $G$-only description and identify the additional collective information required beyond it.

cs.LG · hep-th · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38082

Interpretable Feature Discovery from Unstructured Data Using LLMs and Statistical Choice Models

Baptiste Arnaudo, Amine Bennouna, Sébastien Martin

原文摘要

Feature extraction from unstructured data typically relies on opaque embeddings unsuited to interpretable decision-making. Interpretable features, on the other hand, typically require costly, and often infeasible, hand-engineering. Large language models (LLMs) offer a natural bridge, as they can read raw data and describe it in interpretable terms. One could simply ask an LLM to extract features from a large sample of the dataset, but context limitations and diluted attention can lead to unsatisfactory results. We introduce a method that automatically discovers interpretable, decision-relevant features from unstructured data. Our key idea is to perform pairwise comparisons between items, asking an LLM to articulate, in interpretable terms, the features distinguishing each pair. We treat these explanations, or rationales, as data: we develop a statistical choice model that combines these rationales across the comparisons to identify relevant features and estimate per-item, per-feature scores. We first test our method on a synthetic hiring dataset of cover letters with known ground-truth features and show that it significantly outperforms direct LLM feature extraction in feature recovery. We then apply it to the ChangeMyView (Reddit) debate dataset, where it discovers several interpretable features associated with persuasion while reaching predictive accuracy comparable to established baselines. We also show how the same rationales can be used to audit LLM decisions, yielding a holistic score whose decomposition over interpretable features reveals what the LLM rewards.

stat.ML · cs.AI · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2607.10951

Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion

Pascal Jutras-Dubé, Patrick Pynadath, Jeremy Lu, Yuan Gao, Ruqi Zhang

原文摘要

We introduce sticky jump diffusions (SJDs), continuous-time Markov processes on $\mathbb{R}^d$ whose discrete anchors are token embeddings. In forward time, anchors release their mass at a hazard rate and the released mass diffuses in the continuous ambient space; time reversal couples a score-driven SDE with a sticky jump kernel whose rate and destination are fixed by flux balance with the forward law. We estimate the score and the per-anchor reverse hazards from a single denoising classifier via denoising hazard matching, the hazard analogue of denoising score matching, with simulation-free cross-entropy training. SJD recovers masked diffusion, continuous diffusion, and hybrid diffusion as limits. Its reversal explains features that each family treats as given: the mask of masked diffusion carries no evidence about the source token because the unsticking kernel of every anchor collapses to the same absorbing point; the terminal projection of continuous diffusion is required due to the absence of atoms in its forward marginal, without which flux balance yields no reverse jumps; and the update rules of hybrid diffusion (commit rate, destination, and drift) all follow from flux balance rather than from separate design. Beyond these limits, the unsticking kernel becomes a design space: a cross-position blending corrupts each position toward a blend of its neighbors' clean values or embeddings, turning dependency structure such as spatial locality or a constraint graph into an inductive bias of the corruption itself, and improves over the identity-kernel hybrid on CIFAR-10, Text8, and Sudoku.

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38046

Conformal Predictive Distributions for Contextual Order Fulfillment Optimization

Tinghan Ye, Amira Hijazi, Pascal Van Hentenryck

原文摘要

Ship-to-home fulfillment requires a retailer to select a fulfillment location and carrier before the delivery time is known. These decisions often rely on static transit-time estimates, which cannot represent how delivery-time uncertainty varies across orders and fulfillment options. This paper constructs conformal predictive distributions (CPDs) using Conformal Predictive Systems (CPS) for regression and the cross Venn-Abers predictor (CVAP) for multiclass classification, and uses them in a contextual optimization model for fulfillment decisions. The model is solved by contextual sample-average approximation (C-SAA). The regret of each fulfillment decision, its expected cost above that of the best decision, is bounded by forecast error, scenario-evaluation error, and the candidate-generation gap. The forecast term depends on how well calibration residuals or class probabilities represent the current order. On more than five million proprietary order records, CVAP raises exact-day accuracy from 49.5% for the rule-based estimate to 64.1%. Paired with a decision rule that selects a quantile weighting late errors more heavily, the split CPS (SCPS) gives a point estimate that raises late-delivery detection from 3.8% to 79.4%. Residual-based CPS distributions also meet or slightly exceed nominal interval-coverage targets. In decision experiments with a JD.com simulator over six test dates, C-SAA with CVAP lowers mean total cost by 7.1% and C-SAA with the Mondrian CPS (MCPS) lowers it by 6.7%, relative to C-SAA with the original nonconformal forecast. Both cut the late-delivery rate from 5.7% to below 4.0%.

math.OC · cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2509.19577

MAGIC: Multi-task Gaussian process for joint imputation and classification in healthcare time series

Dohyun Ku, Catherine D. Chong, Visar Berisha, Todd J. Schwedt, Jing Li

原文摘要

Time series analysis has emerged as an important tool for improving patient diagnosis and management in healthcare applications. However, these applications commonly face two critical challenges: time misalignment and data sparsity. Traditional approaches address these issues through a two-step process of imputation followed by prediction. We propose MAGIC (Multi-tAsk Gaussian Process for Imputation and Classification), a novel unified framework that simultaneously performs class-informed missing value imputation and label prediction within a hierarchical multi-task Gaussian process coupled with functional logistic regression. To handle intractable likelihood components, MAGIC employs Taylor expansion approximations with bounded error analysis, and parameter estimation is performed using EM algorithm with block coordinate optimization supported by convergence analysis. We validate MAGIC through two healthcare applications: prediction of post-traumatic headache improvement following mild traumatic brain injury and prediction of in-hospital mortality within 48 hours after ICU admission. In both applications, MAGIC achieves superior predictive accuracy compared to existing methods. The ability to generate real-time and accurate predictions with limited samples facilitates early clinical assessment and treatment planning, enabling healthcare providers to make more informed treatment decisions.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38038

Efficient Deep Gaussian Process Surrogates via Mini-Batch Latent MAP Estimation

Chun-Yi Chang, Chih-Li Sung

原文摘要

Deep Gaussian processes (DGPs) are flexible surrogate models for computer experiments with nonstationary response surfaces, but inference over their latent representations remains computationally challenging. Existing methods typically rely on variational inference or Markov chain Monte Carlo (MCMC), which impose variational-family assumptions or require costly posterior sampling. We propose the latent maximum-a-posteriori deep Gaussian process (LaMP-DGP), which uses gradient-based optimization to obtain MAP estimates of the latent representations and enables efficient probabilistic prediction through linked-GP calculations. To scale LaMP-DGP to large datasets, we develop a mini-batch optimization algorithm and, under a polynomial mini-batch bias-decay condition, establish a stationarity bound for latent-state optimization that motivates a growing batch-size schedule. Across numerical experiments and real-data analyses, LaMP-DGP provides consistently competitive predictive performance relative to doubly stochastic variational inference, stochastic gradient Hamiltonian Monte Carlo, and fully Bayesian Vecchia DGP inference. Computationally, LaMP-DGP is substantially faster than the Vecchia MCMC approach while remaining broadly competitive with the stochastic-gradient alternatives. These results position LaMP-DGP as a competitive and computationally efficient alternative for DGP inference. An R package, "LaMPDGP", is available for implementing the proposed method.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38036

Integral Probability Boundaries

Bogdan Kulynych, Shahab Asoodeh

原文摘要

Bounds based on Integral Probability Metrics (IPMs) are ubiquitous in machine learning and statistics, appearing, e.g., in generalization, alignment, fairness, and privacy. IPMs reduce the discrepancy between two distributions to a single worst-case gap between expectations. This compression is costly: for rare events, safety failures, or under-represented groups, IPM-based bounds are loose or vacuous. We introduce a non-linear generalization of IPMs, which we term Integral Probability Boundaries (IPBs). Unlike IPMs, IPBs provide bounds that are tight at the extremes. We derive their basic properties and present algorithms for estimating them in practically relevant settings. We demonstrate the usage of IPBs in both theoretical and practical applications for alignment, judge-free performance estimation, and fairness, where they enable us to obtain sharp bounds on performance and fairness gaps, e.g., 15 p.p. more precise estimates on fairness gaps for rare groups in a realistic generative modeling task, compared to the state of the art.

cs.LG · cs.IT · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38002

New Multiple Descent Phenomena Induced by Heterogeneous Data

Mark-Anthony Moisescu-Pareja, Elvis Dohmatob

原文摘要

Modern machine learning models are often trained on large datasets assembled from many different sources, whose statistical properties can differ substantially. Understanding how this heterogeneity affects generalization performance is therefore important for explaining how model performance changes as model size increases. In this work, we study this question in linear random-feature models and show that mixtures of datasets with different covariance structures can give rise to a range of multiple-descent phenomena. We derive exact asymptotic expressions for the test error of each data source in the high-dimensional regime, and show how heterogeneity in covariance scale and spectral structure can introduce additional, group-dependent interpolation thresholds as model size increases. These phenomena arise even with noiseless labels and linear activations, showing that neither label noise nor activation nonlinearity is necessary for multiple-descent. Our results therefore provide a different perspective from classical explanations of multiple descent. More broadly, our results have potential implications for algorithmic fairness by showing that model scaling can affect different data groups unevenly, amplifying disparities in their generalization errors. We validate our theoretical results on synthetic Gaussian mixtures and real-world datasets, including MNIST and CIFAR-10.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2510.12975

A Connection Between Score Matching and Local Intrinsic Dimension

Eric Yeats, Aaron Jacobson, Darryl Hannan, Yiran Jia, Timothy Doster, Henry Kvinge, Scott Mahan

原文摘要

The local intrinsic dimension (LID) of data is a fundamental quantity in signal processing and learning theory, but quantifying the LID of high-dimensional, complex data has been a historically challenging task. Recent works have discovered that diffusion models capture the LID of data through the spectra of their score estimates and through the rate of change of their density estimates under various noise perturbations. While these methods can accurately quantify LID, they require either many forward passes of the diffusion model or use of gradient computation, limiting their applicability in compute- and memory-constrained scenarios. We show that the LID is a lower bound on the denoising score matching loss, motivating use of the denoising score matching loss as a LID estimator. Moreover, we show that the equivalent implicit score matching loss also approximates LID via the normal dimension and is closely related to a recent LID estimator, FLIPD. Our experiments on a manifold benchmark and with Stable Diffusion 3.5 indicate that the denoising score matching loss is a highly competitive and scalable LID estimator, achieving superior accuracy and memory footprint under increasing problem size and quantization level.

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.37936

Finite-Step Geometry of Marginal-Preserving Velocity Fields in Generative Flows

Ewsey Obzherin, Valentin Khrulkov

原文摘要

Generative flows can admit different velocity fields that induce the same continuous marginal evolution but behave differently after numerical discretization. We study this finite-step non-equivalence while keeping the marginal path and numerical method fixed. Projected modified equations characterize the component of local integration error that changes the marginal law and give sufficient conditions for higher marginal convergence order. For affine fixed-shape distributions, we characterize exact finite-step reachability and derive spectral obstructions and sharp nearest-reachable Wasserstein errors. For diffusion probability-flow ODEs, we construct skew-score perturbations that preserve the exact marginal path under exact scores and quantify their leakage under score error. Controlled Gaussian and mixture experiments recover the predicted order, reachability, and obstruction phenomena.

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2604.12022

Simulation-Based Inference with Convolutional Maximum Mean Discrepancy for Noisy Data

Ritwik Vashistha, Jeff M. Phillips, Abhra Sarkar, Arya Farahi

原文摘要

Ignoring measurement noise degrades statistical inference. Existing correction techniques are often computationally costly and inefficient. Kernel methods based on maximum mean discrepancy have enabled efficient, likelihood-free inference, but they typically assume clean data. We introduce a framework for inference with samples corrupted by potentially heteroscedastic noise from a known distribution. Central to our method is the convolutional maximum mean discrepancy, which compares distributions after noise convolution and retains metric validity under standard kernel conditions. We establish finite-sample deviation bounds that are unaffected by measurement error and an equivalence between testing under noise and kernel smoothing. Based on this discrepancy, we propose an estimator, establish its consistency and asymptotic normality, and provide an efficient implementation using stochastic gradient descent. We demonstrate the practical effectiveness of our method through simulations and applications in astronomy, anthropometry, and the social sciences. Our method provides a unified, likelihood-free, simulation-based framework for efficient and robust inference for broad classes of problems in noisy data.

stat.ME · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2602.20403

Wasserstein Distributionally Robust Online Learning

Guixian Chen, Salar Fattahi, Soroosh Shafiee

原文摘要

We study online Wasserstein distributionally robust optimization (DRO), where the ambiguity set is a Wasserstein ball centered at the empirical distribution and evolves with each observation. We formulate each round as a zero-sum game and develop an online distributional best-response algorithm that requires only one projected subgradient step per iteration. Combining regret analysis with measure concentration, we establish explicit, instance-dependent, and high-probability convergence rates to the optimal offline DRO value.

cs.LG · math.OC · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.37866

Finite-Sample Optimal Conformal Prediction Sets under Covariate Shift

Zhongyang Li

原文摘要

We study conformal prediction under covariate shift with a known covariate density ratio, seeking prediction sets with minimum expected length under the target distribution. For every calibration sample size, we prove that the rule combining the inverse conditional-density score with randomized weighted conformal calibration minimizes this expected length among all rules valid over the full weighted-exchangeability class. We approximate the oracle using quantile and kernel histograms, including forest-weighted estimators, and select estimators and tuning parameters by resampling to estimate calibrated target length. On a bounded response domain, we bound excess length linearly in $L^1$ density error at fixed calibration size, without requiring a uniform positive lower bound on the true density. We also bound selection-pool and resampling errors. Under explicit growth conditions, the selected procedure approaches the same oracle while the calibration size remains fixed. Experiments on 36 known-density configurations and four public regression datasets compare coverage, prediction-set length, and runtime with existing methods. In targeted low-coverage and strong-shift comparisons, inverse-density selection yields shorter mean sets in most cases and lower mean runtime than the evaluated weighted baselines, with some differences in achieved coverage.

stat.ML · cs.LG · math.CO · math.PR · math.ST

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.37845

An $Ω(T^{3/4})$ lower bound for projection-free online learning with linear minimization oracles

Julien Weibel, Pierre Gaillard, Wouter Koolen, Adrien Taylor

原文摘要

This work considers projection-free online learning algorithms with linear minimization oracles for handling the decision set. For Online Frank--Wolfe (OFW), regret upper bounds were already known for specific OFW tunings [Hazan and Kale, 2012, Hazan, 2016] which scaled as $T^{3/4}$ (where $T$ is the horizon length). However, it remained an open question whether the regret of all OFW tunings scaled as $T^{3/4}$. We answer that question, proving a universal worst-case regret lower bound $Ω(T^{3/4} m^{-1/4})$ for any projection-free algorithm that makes at most $m$ linear minimization oracle calls per round. The analysis suggests an extended OFW to resort to multiple linear optimizations per round, for which we prove a regret upper bound $O(\max \{ T^{3/4} m^{-1/4}, \sqrt{T} \})$.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.37834

Investigating the Early Signatures of Feature Learning in Deep Linear Networks

Shaunak Bhandarkar, Boris Hanin

原文摘要

Deep neural networks learn data-dependent features, a capacity widely credited for their empirical success. However, explicitly characterizing what features emerge -- and when they improve generalization -- is challenging, since training is nonlinear and non-convex and the data is often anisotropic. Here, we give a closed-form account of feature learning for a deep linear network in a mean-field parameterization, trained by one gradient step from initialization on high-dimensional data carrying a low-dimensional (spiked) signal. In the infinite-width mean-field limit, one step updates each layer's feature kernel by a target-aligned rank-one spike, amplified by both the learning rate and the layer depth. We next assess how these features impact prediction through kernel ridge regression (KRR), finding that KRR on the learned feature kernel yields scalar inflation of the KRR estimator at initialization -- an "anti-shrinkage" mechanism that reinflates task-relevant directions suppressed by ridge, while also amplifying noise. To study when this inflation improves generalization, in the overparameterized proportional limit we derive closed-form expressions for the bias, variance, and generalization errors of these estimators in terms of key macroscopic parameters -- notably the spike strength and spike-target alignment. Crucially, we find that a one-step generalization advantage vanishes for isotropic inputs and appears when both the spike strength and an effective signal-to-noise ratio are sufficiently large, demonstrating that a successful first gradient step both requires and benefits from low-dimensional task structure.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.37795

Universal Short-Time Fourier Transform: Data-Adaptive Window-Length Selection for Time-Frequency Analysis

Manan Mittal, Kanad Sarkar, Ryan M. Corey, John R. Buck, Andrew C. Singer

原文摘要

The short-time Fourier transform (STFT) characterizes the local frequency content of time-varying signals by evaluating the discrete Fourier transform over a sliding analysis window. The STFT is governed by a fundamental resolution trade-off: a short window localizes rapid transients in time at the cost of frequency resolution, whereas a long window resolves closely spaced harmonics at the expense of temporal resolution. The Universal Short-Time Fourier Transform (U-STFT) resolves this trade-off sequentially by modeling the analysis window duration as a data-adaptive state variable that indexes the number of hops included in the current observation interval. At each frame, a bank of candidate window durations is scored using normalized order-three Rényi entropy to quantify joint time-frequency concentration. A window length trellis aggregates the resulting $\mathcal{O}(3^M)$ candidate window schedules with linear complexity per frame, guaranteeing a cumulative mix-loss regret bound of $\mathcal{O}((k+1)\log M)$ relative to any hindsight window schedule with $k$ resolution changes over $M$ frames. By linearity of the Fourier transform, posterior expectation across the candidate bank yields a single spectrum equivalent to an STFT evaluated with a data-adaptive effective window, while an envelope-normalized synthesis stream guarantees exact overlap-add reconstruction. Evaluations on synthetic and marine bioacoustic signals demonstrate adaptive resolution tracking, enhanced spectral concentration, and exact time-domain invertibility.

eess.SP · cs.IT · cs.SD · eess.AS · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2602.02491

New explanations and inference for least angle regression

Karl B. Gregory, Daniel J. Nordman

原文摘要

Efron et al. (2004) introduced least angle regression (LAR) as an algorithm for linear predictions, intended as an alternative to forward selection with connections to penalized regression. However, LAR has remained somewhat of a "black box," where some basic behavioral properties of LAR output are not well understood, including an appropriate termination point for the algorithm. We provide a novel framework for inference with LAR, which also allows LAR to be understood from new perspectives with several newly developed mathematical properties. The LAR algorithm at a data level can viewed as estimating a population counterpart "path" that organizes a response mean along regressor variables which are ordered according to a decreasing series of population "correlation" parameters; such parameters are shown to have meaningful interpretations for explaining variable contributions whereby zero correlations denote unimportant variables. In the output of LAR, estimates of all non-zero population correlations turn out to have independent normal distributions for use in inference, while estimates of zero-valued population correlations have a certain non-normal joint distribution. These properties help to provide a formal rule for stopping the LAR algorithm. While the standard bootstrap for regression can fail for LAR, a modified bootstrap provides a practical and formally justified tool for interpreting the entrance of variables and quantifying uncertainty in estimation. The LAR inference method is studied through simulation and illustrated with data examples.

math.ST · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2607.07767

Distributionally Faithful Imputation via Positive Semi-Definite Kernel Density Estimation

Andrea Basteri, Carlo Ciliberto, Alessandro Rudi

原文摘要

Missing values undermine statistical inference and machine learning pipelines, yet most imputation methods rely on heuristics or restrictive parametric assumptions that ignore the joint data distribution. We recast imputation under missing completely at random (MCAR) as density estimation from masked observations: estimate a distribution whose observed marginals exactly match those in the data. Leveraging positive semidefinite (PSD) kernel densities we obtain a convex empirical risk problem with closed form marginals, solvable by a Newton interior point method. We further introduce a non convex extension that supports adaptive hyperparameter selection through alternating minimization. The resulting PSD-Impute model yields both single and multiple imputations from the same fitted density, enjoys statistical consistency, beating the curse of dimensionality for very regular probabilities. Experiments indicate competitive distributional accuracy compared with popular imputation baselines, suggesting strong practical promise.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.37473

QD-PINN:A Physics-Informed Neural Network Approach to Bayesian Quickest Detection in High-Dimensional Diffusions

Hang Ye, Yi Liu, Haipeng Xing

原文摘要

Quickest detection problems (QDPs) arise in a wide range of applications. However, the complementarity structure of the associated variational inequalities poses substantial challenges for traditional numerical methods, particularly in high-dimensional settings. We propose a PINN-based framework for estimating the optimal stopping boundary in QDPs with high-dimensional diffusion processes and spatial dependence, while overcoming the curse of dimensionality. Theoretically, we establish a novel equivalence between the free-boundary PDE and an infinite-dimensional stochastic optimization problem, rigorously justifying the use of finite-dimensional neural networks to approximate the true solution. Furthermore, we assess the robustness of the proposed framework by quantifying uncertainty in the learned stopping boundaries using deep ensembles and Monte Carlo dropout. The results show low variability across independent training runs and stochastic forward passes, indicating stable and consistent boundary estimates. We also compare our PINN results with reference numerical solutions where available and find close agreement.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.37456

An Input Skip Can Reparameterize a VAE Without Improving the Generator

Kevin Lee

原文摘要

An input skip can reduce reconstruction error while leaving a model's distribution of new samples unchanged. For a Gaussian variational autoencoder (VAE) with constant input skip $0\leqα<1$, a model without a skip can be written in skip form by multiplying its decoder output by $b=1-α$ and its noise variance by $b^2$. The distribution of observations and latent variables and its exact generative scores stay the same, while reconstruction mean squared error (MSE) is multiplied by $b^2$. Separate training can produce different generators because optimization depends on the parameter coordinates. In a Gaussian experiment with known density, changing skip and observation variance together reduces MSE but worsens energy score and the estimate of negative log density from importance sampling. On seven annual Deep-ocean Assessment and Reporting of Tsunamis (DART) records, energy score differences at the same observation variance have both signs on four original records and favor the skip on three records from 2018. Probabilistic principal component analysis (PPCA) has lower mean energy scores than every neural DART fit. With the encoder and decoder kept the same, estimating observation variance from training reconstruction error can also worsen energy score. Assessing generation requires more than reconstruction error.

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.37450

Graph Representation Learning via Adaptive Expectile Aggregation

Tianyi Zhu, Wei Lan

原文摘要

Graph neural networks (GNNs) rely critically on neighborhood aggregation, yet conventional mean aggregation may fail to capture class-relevant distributional information, particularly in heterophilic graphs. We propose ExpectileGCN, an adaptive graph aggregation framework that introduces a learnable expectile level to capture neighborhood information beyond the mean. The proposed operator continuously adjusts aggregation between minimum- and maximum-type behaviors, with mean aggregation as a special case. A differentiable approximation based on sigmoid smoothing and fixed-point iterations enables end-to-end optimization. Theoretically, we establish stability properties and transductive generalization guarantees for ExpectileGCN. We further investigate its classification risk under heterophilic neighborhood models. For informative neighborhoods, we show that expectile aggregation can exploit class-dependent scale differences to achieve a classification advantage over mean aggregation. Under a local asymptotic regime with diminishing per-neighbor label informativeness, we establish that this advantage can persist in terms of limiting classification risk. Experiments on homophilic and heterophilic benchmark graphs demonstrate strong predictive performance and improved robustness under feature corruption and structural perturbations.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2604.14809

Expert-Guided Class-Conditional Goodness-of-Fit Scores for Interpretable Classification with Informative Missingness

Shahar Cohen, David M. Steinberg, Yael Radzyner, Yochai Ben Horin

原文摘要

We study a classification problem with three key challenges: pervasive informative missingness, the integration of partial prior expert knowledge into the learning process, and the need for interpretable decision rules. We propose a framework that encodes prior knowledge through an expert-guided class-conditional model for one or more classes, and use this model to construct a small set of interpretable goodness-of-fit features. The features quantify how well the observed data agree with the expert model, isolating the contributions of different aspects of the data, including both observed and missing components. These features are combined with a few transparent auxiliary summaries in a simple discriminative classifier, resulting in a decision rule that is easy to inspect and justify. We develop and apply the framework in the context of seismic monitoring used to assess compliance with the Comprehensive Nuclear-Test-Ban Treaty. We show that the method has strong potential as a transparent screening tool, reducing workload for expert analysts. A simulation designed to isolate the contribution of the proposed framework shows that this interpretable expert-guided method can even outperform strong standard machine-learning classifiers, particularly when training samples are small.

stat.ML · cs.LG · stat.AP

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2503.04956

Foreclassing: From Forecasts to Decisions

Daniel Andrew Coulson, Martin T. Wells

原文摘要

Time series forecasts are widely used to inform decisions under uncertainty. In many applications, these decisions involve classifying a time series based on outcomes that have yet to be observed. In this paper, we introduce Foreclassing, a new machine learning problem in which the class label of a time series is defined as a function of its future observations. We provide a mathematical formulation of Foreclassing and establish several theoretical results characterizing the problem. These results describe how additional future information affects Bayes classification risk, provide bounds on the classification error of learned Foreclassing models, and demonstrate how future observations used during training can yield finite-sample improvements over conventional classifiers. To provide a benchmark for future research, we propose ForeClassNet, a family of neural network models that jointly perform forecasting and classification while accounting for uncertainty in future outcomes. We investigate a variety of model architectures and evaluate their performance against conventional time series classifiers and forecasting-then-classify approaches. Experiments on datasets spanning weather, energy, and finance demonstrate the potential of Foreclassing to improve classification performance, probabilistic prediction, and downstream decision-making. Together, these theoretical and empirical contributions establish a foundation for future research on Foreclassing.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.09552

Phases of Muon: When Muon Eclipses SignSGD

Elliot Paquette, Noah Marshall, Lucas Benigni, Guangyuan Wang, Atish Agarwala, Courtney Paquette

原文摘要

Muon and related spectral optimizers have recently demonstrated strong empirical performance as scalable stochastic methods, often outperforming Adam. Yet their behavior remains poorly understood. We analyze stochastic Muon, which orthogonalizes an exponential moving average of minibatch gradients, on a high-dimensional matrix-valued least squares problem, and we compare it with stochastic SignSGD, a simple baseline comparable to Adam. We derive explicit deterministic dynamics that provide a tractable framework for studying learning behavior. We show that Muon has non-linear sensitivity to the effective batch $B_{\rm eff}=B(1+β)/(1-β)$. With exact whitening and small $B_{\rm eff}$ Muon fails to resolve smaller eigenmodes, slowing down convergence. Momentum and finite Newton--Shulz iterations can act as a ridge which lessen this effect. This behavior is in contrast with SignSGD which has no transition, leading to different optimal learning rates, but with matching convergence rate up to a constant factor in the isotropic setting. For anisotropic data, large batch setting Muon performs a square-root preconditioning with respect to the data covariance spectrum while SignSGD provides no preconditioning for generic covariances. We analyze time to target loss in a power law covariance model and derive a phase diagram in the data exponent $α$ and target exponent $ζ$. Muon is favored for steep covariance spectra with rich targets ($α>2$, $ζ<α+1$), SignSGD is favored for flat spectra ($α<1$), and in other regions the winner depends on how the target loss scales with dimension. Our results shed light on why replacing Adam with Muon is not trivial even from a pure optimization perspective.

math.OC · cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.37398

An end-to-end framework for uncertainty-aware validation of generative models

Giada Badaracco, Sean Benevedes, Christina Reissel, Philip Harris, Gaia Grosso

原文摘要

Density estimation with generative models is a common task in the physical sciences, for example in background estimation in particle physics. Existing machine learning techniques, however, lack methods to quantify uncertainties that can be propagated into a statistical test. That limits the validation of generative models to standard goodness-of-fit tests, in which the generative model is treated as exact (point-null hypothesis). But a model fitted to a finite training set is never exact, so if its uncertainties are neglected, any sufficiently powerful goodness-of-fit test will eventually reject it once the test set is large enough to resolve this estimation error. We propose SIGMA, an end-to-end framework combining density estimation with principled uncertainty quantification. We use Gaussian mixture models, for which uncertainties are tractable, and further construct a new goodness-of-fit test in which these uncertainties are profiled as nuisance parameters to account for the statistical error of the fit, promoting the conventional point-null test to a composite one. A rejection then signals genuine mismodeling rather than a test set large enough to resolve the error of the fit, provided the propagated uncertainties faithfully cover that error. We demonstrate the framework on a controlled two-dimensional toy model and on a collider physics benchmark modeling the QCD background process. With uncertainties included, the same model can be trusted on test sets up to an order of magnitude larger than the standard test allows. Our work thus provides a statistically grounded basis for a robust use of generative models in the physical sciences.

stat.ML · cs.LG · hep-ex · hep-ph

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2502.04309

Targeted Learning for Inference on Data Fairness

Joseph H. Rudoler, Alexander Asemota, Giles Hooker

原文摘要

As algorithmic decision making has become increasingly ubiquitous even in high-stakes settings, significant effort has been invested in defining, detecting, and eliminating unfair model-driven outcomes. Prior work has largely focused on estimating the fairness properties of a given predictive model, and quantifying uncertainty based on variability in a fixed model's performance across evaluation datasets. Alternatively, we can also ask about uncertainty that the model inherits from its training data; here we evaluate data fairness - fairness in the data generating process used for training. We perform inference on data fairness using targeted learning, a flexible framework for nonparametric inference, and derive estimators for demographic parity, equal opportunity, and conditional mutual information. Simulations show that our data fairness estimators improve confidence-interval coverage and reduce false detections of population unfairness relative to fixed-model inference. We also demonstrate their application to real data.

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.37390

LinMix: Data Mixing as Linear Domain Attribution

Hubert Baniecki, Przemyslaw Biecek, Martin Pawelczyk

原文摘要

Language model performance depends strongly on the mixture of domains in the pretraining data corpus. Popular data mixing methods select these domain weights by training many small proxy models on random mixes and then fitting a regression model to predict performance from the mix. We recognize this practice as a domain attribution problem, where the regression measures each domain's contribution to the target metric. Through this lens, the standard Dirichlet sampling of proxy data mixes is a poor regression design that requires tuning and cannot support a linear fit for hundreds of domains. Building on classical optimal experimental design, we introduce LinMix, which samples mixes at the vertices of the capped simplex so that each domain is at its cap or absent. This hyperparameter-free vertex sampling decorrelates domain effects, enabling an interpretable linear model whose coefficients are the domain attributions. The optimal mix given by LinMix provably keeps only domains with positive attribution values and requires no search over candidate mixes. We further establish the optimality of vertex sampling, and prove that Dirichlet sampling inflates the attribution error when its mixes concentrate around the prior data mix. Pretraining experiments with 1B-parameter OLMo2 and 4B Qwen3.5 language models show that LinMix outperforms Dirichlet-based data mixing across reasoning, code, and math benchmarks, with relative gains of 46% on MBPP and 14% on GSM8K. At 288 fine-grained domains, only vertex sampling supports a linear fit, and LinMix improves benchmark accuracy by 11% on average over the best nonlinear baseline.

cs.LG · cs.CL · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.37737

Anisotropic Trust Region Warping for Prior-Data Fitted Networks in Bayesian Optimisation

Samuel Younger, Tinkle Chugh, George De Ath

原文摘要

Prior-Data Fitted Networks (PFNs) replace the hyperparameter refitting of Gaussian Process (GP) surrogates in Bayesian optimisation with a single forward pass, at the cost of a prior that is fixed at meta-training time. We show empirically that a PFN's predictive log-likelihood falls increasingly behind that of a refitted GP as the objective function's lengthscale grows beyond the mode of the PFN's meta-training prior, and that this deficit carries over to optimisation performance. To correct the mismatch without retraining the network, we introduce Anisotropic Trust Region (ATR) warping. Per-dimension lengthscales are estimated by kernel target alignment, and an affine rescaling of the input space maps the objective onto the lengthscale the PFN expects, with a trust region around the anchor determining which observations are retained, leaving the PFN's $\mathcal{O}(d \cdot N^2)$ inference cost unchanged. On functions drawn from a Matérn-3/2 kernel, ATR warping improves optimisation performance of the PFN at both extremes of the lengthscale range. Our implementation is publicly available at https://github.com/youngers2006/Trust-Region-Warping-For-PFNs.

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗