跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+0新增论文
+0其中 AI 相关
+1已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-08 18:41 · 全站最近入库：2026-11-08 06:02各信源进度 →

统计口径与数据新鲜度

统计 2026-11-07 19:19 — 2026-11-08 19:19（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-08 18:41最近发现新增版本：2026-11-08 06:02库内论文最新更新时间：2026-11-06 02:59

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

17,100 已收录论文14,661 AI 相关4,057 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 460 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-08 18:39（北京时间）
版本更新 · v7arXiv:2509.25783

Sharpness of Minima in Deep Matrix Factorization

Anil Kamber, Rahul Parhi

原文摘要

Understanding the loss-landscape geometry near a minimum is key to explaining the implicit bias of gradient-based methods in non-convex optimization problems such as deep neural network training and deep matrix factorization. A central quantity for characterizing this geometry is the maximum eigenvalue of the Hessian of the loss. Currently, its precise role has been obfuscated because no exact expressions for this sharpness measure were known in general settings. In this paper, we present the first exact expression for the maximum eigenvalue of the Hessian of the squared-error loss at any minimizer in deep matrix factorization problems, resolving an open question posed by Mulayoff and Michaeli [2020]. This expression reveals a fundamental property of the loss landscape in deep matrix factorization: It is sufficient, but not necessary, for flatness that, across all layers, the product of the spectral norms of the left and right intermediate factors equals a constant determined by the depth and the spectral norm of the target matrix. Most notably, in both depth-2 matrix factorization and deep overparameterized scalar factorization, we show that this condition is both necessary and sufficient for flatness, which implies that flat minima are spectral-norm balanced and not necessarily balanced in depth-2 matrix factorization. To complement our theory, we provide empirical verification, relying on our exact expression, of an escape phenomenon during gradient-based training near a minimizer of a deep matrix factorization problem.

stat.ML · cs.LG · math.OC

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.39043

Modular Bayesian Foundation Models: Bayesian time series encoders for OOD detection, routing and forecasting

Mame Diarra Toure, Quang Hung Pham, Mohamed Ryad Zemouri, David Stephens, Luc Vouligny, Martin Gagnon

原文摘要

Industrial systems continuously generate time series that vary in sampling frequency, scale, dynamics, and operating regime. Consequently a single time-series foundation model may not be able to represent all regimes effectively and retraining a large model for every new dataset or regime shift is computationally expensive. Modular foundation models address this issue by reusing specialised encoders trained on diverse datasets and by allowing the expansion of capabilities through the addition of new experts. This approach requires compact experts to minimize storage cost and a proper selection mechanism of experts for unseen data. We propose a modular foundation model framework that combines end-to-end Bayesian encoder training with uncertainty based routing. Using Singular Bayesian Neural Networks (SBNNs), we build a collection of compact Bayesian encoders whose epistemic uncertainty guides label-free selection. At the selected ranks, our Bayesian collection uses 63.8\% less storage than its deterministic counterpart. We evaluate this framework on time-series data through distribution shift, encoder selection, and probabilistic forecasting. Across seven incoming datasets, uncertainty-based routing is competitive with distance-based routing in terms of forecast MSE, while reducing average ECE from 0.086 to 0.072 and average NLL from 0.907 to 0.836. It also increases PICP90 from 82.28\% to 89.03\%. In a separate model-family comparison, we evaluate the SBNN pipeline against full-rank Bayes by Backprop, deep ensembles, and MC Dropout. The SBNN pipeline achieves an average MSE of 0.390, comparable with 0.396 for deep ensembles, 0.389 for MC dropout, and 0.443 for the full-rank Bayesian model. It uses 63.9\% fewer parameters than MC dropout, 81.9\% fewer than the full-rank model, and 92.8\% fewer than the deep ensemble.

stat.ML · cs.LG · stat.AP · stat.CO

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2510.04421

Learning Survival Models with Right-Censored Reporting Delays

Yuta Shikuri, Hironori Fujisawa

原文摘要

Survival analysis provides statistical methods to model the time until an event occurs. Reporting delays arise when event times are not observed at their occurrence but are only revealed upon reporting. This issue is particularly critical for timely risk evaluation when the observation window is short due to administrative censoring. In this study, we incorporate right-censored reporting delays by jointly modeling parametric hazards for the event and reporting processes. We then construct a consistent estimator for the model parameters and develop a Monte Carlo expectation-maximization algorithm to compute it. To address the challenges posed by administrative censoring, we leverage these findings and propose a transfer-learning procedure. Experimental results demonstrate that our method improves the accuracy of timely risk evaluation under administrative censoring.

stat.ML · cs.LG · math.ST

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38999

A Spectral Boundary for Independence Learnability

Senhan Yao, Yuxin Zhou

原文摘要

Learning independence is fundamental to statistical inference and machine learning, yet learning independence between entire time series by a test remains challenging when only a single observation record is available. Existing guarantees often rely on structural or mixing assumptions, while stationarity and ergodicity alone do not permit consistent testing. In this work, we identify a class-level boundary governed by shared nontrivial Koopman eigenvalues, which represent common persistent oscillatory frequencies of the two processes. When no such eigenvalues are shared, we extend shift-based rank calibration with finite-window dependence witnesses and a data-adaptive threshold to construct a deterministic, weakly consistent test with finite-sample false-positive control, without quantitative mixing assumptions. Conversely, we show that no possibly randomized test can achieve vanishing false-positive probabilities under every independent jointly totally ergodic law while consistently detecting every dependent jointly totally ergodic law with a shared nontrivial eigenvalue. This impossibility already holds for binary observations and persists within any prescribed nonempty open eigenvalue window, identifying a spectral obstruction to universal learning of full-process independence.

math.ST · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38965

Orthogonal Inference for Conditional Z-Estimation without Cross-Fitting: A Distributional Nearest-Neighbor Approach

Jakob R. Juergens

原文摘要

Estimating conditional parameters, such as heterogeneous treatment effects, often requires rich covariate adjustment, even when the parameters are reported along only a few dimensions. Fitting nuisance functions on the full sample makes all observations available for adjustment, but creates dependence between fitted scores and the observations at which they are evaluated, which is typically avoided by cross-fitting. We develop pointwise inference with full-sample nuisance predictions for exactly identified conditional moment equations, recovering conditional average treatment effects as a special case. We localize the estimating equations around the reporting point using distributional nearest neighbors (DNN), a U-statistic-based averaging procedure. Under sufficient conditions on the local design, orthogonality, prediction accuracy and quantitative deletion stability, the estimator is asymptotically normal, and a frozen-score jackknife consistently estimates its limiting covariance with nuisance predictions held fixed. In simulations, full-sample and cross-fitted inference perform similarly, with coverage shortfalls from nuisance approximation error, spatial bias and heavy studentized tails. An application to 401(k) eligibility using Survey of Income and Program Participation data finds similar estimates across nuisance prediction methods, with wide normal-reference intervals.

econ.EM · math.ST · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2609.23290

Stochastic Flow Map for Count Data

Ganchao Wei

原文摘要

High-dimensional count data are common in scientific applications, but most diffusion and flow models are designed for continuous or categorical data, and generation often requires many sequential model evaluations. Here, we propose Count Flow Map, a generative model that learns finite-time transitions directly in count space for one- or few-step generation and transport between count distributions. Our model directly learns stochastic transitions over finite time intervals, using Poisson births and Binomial deaths to preserve nonnegative integer counts without a predefined maximum. These transition models are trained to match the underlying local birth--death dynamics and to maintain consistency across step sizes. We characterize the connection between local dynamics and finite-time transition consistency and derive a bound on the generation error. After validating Count Flow Map in several simulations, we apply it to single-cell drug perturbation prediction and neural population forecasting, where it captures perturbation responses and supports forecasts of high-activity events with only one or a few model evaluations. Together, these experiments demonstrate that Count Flow Map enables high-quality generation directly in count space across inference budgets, from one-step to few-step generation, using a single trained model.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.24246

CoCoNet: Convex Conformal Network for Multi-Dimensional Uncertainty Regions

Alireza Moradi, Thomas Bruys, Pascal Van Hentenryck, Reza Zandehshahvar

原文摘要

Conformal prediction (CP) provides distribution-free coverage guarantees under exchangeability, but the efficiency of its prediction regions depends critically on the choice of the conformity score. This is particularly important in multivariate regression, where dependencies across output dimensions induce complex joint uncertainty structures. This paper introduces CoCoNet (Convex Conformal Network) for learning convex conformity scores that induce flexible, data-adaptive prediction regions. CoCoNet learns the score from held-out data by optimizing prediction-region efficiency while enforcing convexity with respect to the residual. CoCoNet then applies split-conformal calibration to retain finite-sample coverage guarantees. The paper presents two variants of the method: CoCoNet$_{\mathrm{Spline}}$, based on piecewise-linear scores, and CoCoNet$_{\mathrm{ICNN}}$, based on Input Convex Neural Networks. Experiments compare CoCoNet with other CP baselines over synthetic and real-world datasets with output dimensions ranging from 2 to 16. The results demonstrate the flexibility of CoCoNet in learning different response types. Across 8 models and 30 multi-dimensional regression tasks, CoCoNet$_\text{ICNN}$ and CoCoNet$_{\mathrm{Spline}\text{-}\ell_1}$ achieve the best efficiencies with average ranks of 1.45 and 2.75, respectively. In particular, CoCoNet$_\text{ICNN}$ ranks first on 22 tasks out of 30 in terms of efficiency.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38938

Memorization Capacity of Invariant Convolutional Neural Nets

Sjoerd Dirksen, Patrick Finke, Paul Geuchen

原文摘要

Neural networks are powerful memorizers which are often able to interpolate their training data even when the labels have been randomized. While there exists an extensive literature on the memorization capacity of fully connected networks, in practice more specific architectures with problem-dependent properties are preferred, such as convolutional networks for image classification. In this paper we consider convolutional networks which are invariant with respect to a given arbitrary finite group of orthogonal transformations, which may include translations, reflections, and/or rotations of images and invariances of higher order tensors. We prove upper bounds on the size of an invariant convolutional neural network with threshold activations that is required to interpolate a given dataset with two classes. Our bounds are formulated in terms of a complexity measure that takes the invariant structure into account and is instance-specific, i.e., it is related to the mutual arrangement of the classes in the given dataset and their geometric properties rather than the number of samples. Our proof is constructive: for given input data, instances of an interpolating network can be computed via a randomized algorithm with high probability. In numerical experiments we investigate the practical effectiveness of this algorithm.

stat.ML · cs.LG · math.ST

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.14301

Language-Induced Priors for Domain Adaptation

Qiyuan Chen, Jiayu Zhou, Raed Al Kontar

原文摘要

Domain adaptation faces a fundamental paradox in the cold-start regime. When target data is scarce, statistical methods fail to distinguish relevant source domains from irrelevant ones, which often leads to negative transfer. In this paper, we address this challenge by leveraging expert textual descriptions of the target domain, a resource that is often available but overlooked. We propose a probabilistic framework that translates these semantic descriptions into a choice model, namely a Language-Induced Prior (LIP), that learns the preferences from a pretrained Large Language Model. The LIP is then integrated into an Expectation-Maximization algorithm to identify source relevance. Methodologically, this framework is compatible with any parametric model where a likelihood is available. It allows the LIP to guide the selection of sources when target signals are weak, while gradually refining these choices as samples accumulate. Theoretically, we prove that the estimator roughly matches an oracle cold-start mean squared error under a correct prior, while remaining asymptotically consistent regardless of the quality of the LIP. Empirically, we validated the framework on a descriptive (Gaussian estimation), a predictive (C-MAPSS dataset), and a prescriptive task (MuJoCo Hopper).

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2610.12094

Differentiable Systematic Resampling for Variational Sequential Monte Carlo

Fredrik Cumlin, Saikat Chatterjee

原文摘要

Particle filters are a standard tool for nonlinear state estimation, but their resampling step is discrete, preventing gradient-based learning in variational sequential Monte Carlo. We introduce Differentiable Systematic Resampling (DSR), a temperature-controlled relaxation of systematic resampling, that preserves the CDF-ordered, banded structure of systematic resampling while enabling full gradient flow. DSR converges to exact systematic resampling as the temperature vanishes, and we prove a pointwise exponential convergence rate for the induced bias. Compared to optimal-transport-based differentiable resampling, DSR avoids iterative solvers and has substantially lower computational overhead. Experiments on stochastic dynamical systems and real-world handwriting data show that DSR achieves comparable or superior filtering and dynamics learning performance.

stat.ML · cs.LG · eess.SP

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38797

Pinpointing Pivot Points: The DECIDE Framework for Data Analysis Decision Importances

Zachary T. Rewolinski, Jan Simson, Frauke Kreuter, Christoph Kern, Bin Yu

原文摘要

Human judgment calls made throughout the data science life cycle can change the conclusions drawn from data. Independent analysts often make different choices when given the same data, leading to often unacceptable variations in results and contributing to the ongoing reproducibility crisis across scientific disciplines. The rapid adoption of agentic tools threatens to amplify this crisis. Several approaches have been developed to address this variation by following the same strategy: repeating an analysis under different combinations of analytical decisions and inspecting the distribution of outcomes. Examples include multiverse analysis, specification curve analysis, and vibration of effects. More broadly, the Predictability-Computability-Stability (PCS) framework assesses how documented judgment calls across the data science life cycle affect conclusions, with predictability serving as a reality-check that a model captures signal. Each of these approaches results in a "multiverse" object. While the distribution of outcomes reflects the overall variability of conclusions, estimating the influence of each individual decision requires additional tools. A common heuristic is to train a model to predict the outcome from these decisions and treat its coefficients or feature importances as proxies for each decision's influence. Although this strategy may work in certain cases, it assumes the model is correctly specified and accurately predicts the outcome, which we show is not always the case. In this work, we introduce DECIDE, a model-free framework that quantifies each decision's importance from the actual outcomes across the plausible decision space, avoiding issues of predictability by design. We demonstrate the effectiveness of DECIDE by applying it to real-world multiverse analyses. We further provide what is, to our knowledge, the first formal definition of the multiverse.

stat.ML · cs.LG · stat.ME

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38753

Metropolized backward sampling for conditional sequential Monte Carlo

Adrien Corenflos

原文摘要

Particle Gibbs samplers with backward sampling are widely used for sampling the smoothing distribution of state-space models, but, with few particles, the new trajectory often coincides with the current one at many time steps. A known but rarely used remedy replaces the random selection of the final particle by a Metropolis--Hastings move which avoids the current particle. We show that, with backward sampling, this remedy only helps at the last time steps, because its effect on the update rates decays geometrically with the distance to the final time. We propose instead Metropolized backward sampling, which makes such a move at every step of the backward pass, forcibly switching trajectories whenever possible. Additionally, we introduce a coupled version which prevents proposing the current trajectory, maximizing the chances of an actual switch versus the current trajectory and performing perfect sampling when the state-space model has no observations, a feature not shared by backward sampling. We prove that, at every time step, both methods dominate backward sampling as they update the state more often and have smaller lag-one autocorrelations than it does. Our methods are drop-in replacements for backward sampling, and can be used to improve any particle Gibbs sampler.

stat.CO · eess.SP · math.PR · stat.ME · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38729

Integrating multi-source local information for global undirected graph reconstruction via integer programming

Nir Keret, Armeen Taeb, Ali Shojaie

原文摘要

Graphical models are widely used to represent conditional independence structures, but learning them is challenging when data are fragmented across sources that measure different subsets of variables and may not be directly shareable. We develop a general framework for reconstructing a shared global graph from heterogeneous, partially observed sources. Our framework permits both the observed variable sets and source-specific distributions to differ. Under a source-wise faithfulness condition, local conditional independence graphs correspond to latent projections of the shared global graph, which is identifiable up to the equivalence class induced by the observed variable sets. The framework integrates raw data, estimated source graphs, or a combination of both, accommodating settings with restricted access to individual-level data, such as privacy constraints. We develop mixed-integer optimization formulations and scalable approximate algorithms for several classes of graphical models. Unlike standard high-dimensional analyses, our guarantees are governed by new combinatorial notions of signal strength that capture how information propagates across partially observed sources. Extensive simulations further show that integration can substantially improve estimation of source-specific projected graphs relative to separate source methods. Real-data application illustrates the framework's use with partially-observed sources of fatty acid measurements.

stat.ME · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38686

On the Generalization Gap of Streaming Federated Learning with Markovian Data

Masoud Kavian, Milad Sefidgaran, Youssef Attia El Hili, Malik Tiomoko, Abdellatif Zaidi

原文摘要

Streaming federated learning (SFL) allows clients to train a shared model from continuously arriving data without retaining large local datasets. Such streams often exhibit temporal dependence, yet its effect on generalization remains poorly understood. In this paper we study the expected generalization gap of the aggregated model when client data follow Markov processes, using information-theoretic and rate-distortion techniques. Our framework accommodates heterogeneous Markovian dynamics across clients and distinguishes temporal dependence from information acquired during local learning. We establish bounds under cross-entropy loss, specializing them to Linear Next-Token Prediction (LNTP) and Self-Attention Next-Token Prediction (SANTP). These results quantify the effects of client count, communication rounds, sample size, data dimension, mixing time, and context length. In particular, they reveal a communication--generalization tradeoff: although frequent aggregation may help optimization, it can weaken generalization guarantees. They also suggest a potential advantage of distributed over centralized streaming learning under a fixed total data budget. Temporal dependence enters through the square root of the mixing time. By analyzing the history-state process, we obtain refined bounds with improved context-length dependence under suitable mixing conditions. Experiments on GPT-style language-model pretraining with TinyStories and real-world time-series forecasting illustrate how collaboration and communication frequency influence generalization under temporal dependence

cs.LG · cs.IT · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2510.23254

Optimal In-context Adaptivity and Distributional Robustness of Transformers

Tianyi Ma, Tengyao Wang, Richard J. Samworth

原文摘要

We study in-context learning problems where a Transformer is pretrained on tasks drawn from a mixture distribution $π=\sum_{α\in\mathcal{A}} λ_α π_α$, called the pretraining prior, in which each mixture component $π_α$ is a distribution on tasks of a specific difficulty level indexed by $α$. Our goal is to understand the performance of the pretrained Transformer when evaluated on a different test distribution $μ$, consisting of tasks of fixed difficulty $β\in\mathcal{A}$, and with potential distribution shift relative to $π_β$, subject to the chi-squared divergence $χ^2(μ,π_β)$ being at most $κ$. In particular, we consider nonparametric regression problems with random smoothness, and multi-index models with both random smoothness and random effective dimension. We prove that a large Transformer pretrained on sufficient data achieves the optimal rate of convergence corresponding to the difficulty level $β$, uniformly over test distributions $μ$ in the chi-squared divergence ball. Thus, the pretrained Transformer is able to achieve faster rates of convergence on easier tasks and is robust to distribution shift at test time. Finally, we prove that even if an estimator had access to the test distribution $μ$, the convergence rate of its expected risk over $μ$ could not be faster than that of our pretrained Transformers, thereby providing a more appropriate optimality guarantee than minimax lower bounds.

stat.ML · cs.LG · math.ST

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38559

A goodness of fit approach to select the optimal visit sequence for nonparametric data synthesis

R{é}my Chapelle, Bruno Falissard, Mohammed Sedki

原文摘要

Synthetic data are a promising way to maintain privacy while disseminating research data, thereby enabling compliance with the open science principles. Popular methods for generating synthetic data include sequential synthesis, where each variable is generated conditionally on the preceding ones in a given order. This order, also called the visit sequence of the synthesizer, is known to affect the quality of synthetic data. This raises the question of the criteria that should guide its choice. In this work, we present a novel method to perform such selection based on a goodness of fit criterion, and show that it outperforms alternative structure-based methods, both in terms of synthesis quality and time complexity. The proposed method also has the advantage of being applicable to any type of data and any class of conditional distribution models.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2609.27836

Poincaré's Inequality on Sliding-Window Occupation Count Kernels and the Optimal Normalized Spectral-Gap Constant

Yanjin Xiang, Zhihua Zhang

原文摘要

Let $P$ be an irreducible reversible Markov kernel on an $m$-state space $Ω$, and denote its right spectral gap by $\Gap(P) =1-λ_2(P)$. For a stationary trajectory, let $K_t$ be the occupation-count vector of the length-$n$ window beginning at time $t$. The stationary pair $(K_0,K_1)$ defines a reversible projected count kernel $\widetilde P_n$. For each fixed $m\ge2$, define \[ c_m^\star= \inf_{\substack{|Ω|=m,\ P\ \mathrm{irreducible\ and\ reversible} n\ge2}} \frac{n\Gap(\widetilde P_n)}{\Gap(P)}. \] We prove \[ \frac1{1080m}\le c_m^\star\le q_{m-2}, \qquad q_0=\frac14,\quad q_{r+1}=q_r(1-q_r). \] We also show $q_{m-2}=(m+\log m+O(1))^{-1}$, which implies that $c_m^\star=Θ(m^{-1})$. Thus, the reciprocal optimal comparison coefficient $C_m^{\mathrm{opt}}=(c_m^\star)^{-1}$ is of the order $m$. In the two-state case, we determine the exact values $c_2^\star=1/4$ and $C_2^{\mathrm{opt}}=4$. %the exact constants for $m\ge3$ remain open. The lower bound also holds for $n=1$ and is uniform in $P$, including sparse and periodic kernels. The proof combines short-window decorrelation with an averaged anchor-excursion decomposition, a Green-kernel hitting estimate, and a stopped Carleson--Hardy inequality. A nested rare-state construction produces finite $m$-state witnesses whose normalized Rayleigh quotients approach $q_{m-2}$ through an ordered sequence of limits. For every fixed finite irreducible reversible aperiodic kernel on at least two states, $\Gap(\widetilde P_n)=Θ_P(n^{-1})$.

math.ST · math.PR · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2606.10734

SPACR: Single-Pass Adaptive Training of Uncertainty-Aware Conformal Regressors

Soundouss Messoudi, Sylvain Rousseau, Sébastien Destercke

原文摘要

Conformal Prediction (CP) provides robust uncertainty guarantees for predictive models, but is typically applied post hoc, which misaligns model training with the conformal goal of producing efficient (i.e., narrow) intervals. We propose SPACR (Single-Pass Adaptive Conformal Regressor), a novel method for directly training uncertainty-aware regressors within a differentiable loss. SPACR jointly optimizes accuracy, efficiency, and validity without batch-splitting or a predefined confidence level during training. As a result, a single SPACR model yields valid prediction intervals at multiple confidence levels during inference, avoiding the costly retraining required by methods like Directly Optimized Inductive Conformal Regression (DOICR). Experiments on diverse tabular and image datasets show that SPACR consistently gives tighter intervals and better coverage-efficiency trade-offs compared to standard CP and DOICR, while significantly reducing computational costs relative to retraining-dependent baselines.

cs.LG · stat.ME · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2507.14492

Verifying Glitches in Decision Tree Ensembles

Satyankar Chandra, Harsh Chaurasia, Ashutosh Gupta, Kaushik Mallik, Krishna Shankaranarayanan, Namrita Varshney

原文摘要

Decision tree ensembles are widely used for tabular prediction tasks in judiciary, healthcare, banking, etc. We show that the piecewise-constant structure of these models can create narrow local oscillations, leading the ensemble score to sharply drop and then rise (a downward spike), or sharply rise and then drop (an upward spike), over a small change in a single feature. We call such patterns glitches. A glitch is not, by definition, an error. Rather, it is a formally checkable audit target requiring inspection. Formally, glitches are witnessed by ordered input triples in which a small monotonic change in a single input feature causes the output to first change in one direction and then revert in the opposite direction. Through a substantial experimental study, we show that glitches have widespread existence in even the state-of-the-art models performing various critical tasks. Our findings also indicate that glitches enrich and extend existing notions like robustness and monotonicity, providing us a more refined tool to isolate anomalies for inspection. On the theoretical side, we prove that deciding whether a decision tree ensemble contains a glitch is NP-complete, even for bounded-depth trees. We then present a mixed-integer linear programming encoding for the global and local glitch verification problems, and implement our algorithm in a verifier for gradient-boosted decision trees. Experiments show our verifier can scale to realistic model sizes from the literature.

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38437

Capacity of the Hopfield-Hebbian dynamics with correlated patterns

Mihailo Stojnic

原文摘要

Allowing for small fraction of errors in retrieved stored binary patterns, Hopfield-Hebbian neural network (NN) associative memory dynamics [14] achieves linearly scaled capacity. The famous proportionality constant $α_c\approx 0.14$ is obtained in a large dimensional scenario as the ratio of the number of reliably stored patterns $m$ and the patterns' length $n$ under the assumption that uniformly randomly chosen binary patterns are \emph{independent} among themselves. While such an idealized scenario is now well understood, more practical scenarios with correlated/noisy datasets remaining challenging even for classical statistical mechanics replica approaches. We consider precisely such practical settings where each \emph{archetype} pattern is replaced with $M$ correlated \emph{examples} with parameter $r$ governing the quality of correlation. We develop a powerful Random Duality Theory (RDT) based framework to analyze correlated data and already on the first level of RDT lifting obtain capacity characterizations that remarkably well match behavior exhibited through simulated dynamics. Moreover, characterizations explicitly feature correlation parameters $M$ and $r$, which allows to precisely determine their effects on phase-transitioning behavior of the underlying dynamics. Our results are also a well-fitting complement to the existing literature. Namely, to circumvent challenging statistical mechanics analyses, an alternative combination of spectral and replica methods is considered in [4]. We confirm that [4]'s elegant spectral characterizations are excellent approximations in the low correlated regime. At the same time, we also close the gap in the regimes with higher correlation levels where spectral methods deviate from the practical dynamics performance.

stat.ML · cond-mat.dis-nn · cs.IT · cs.LG · math.PR

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2607.11653

Bet on Features: Anytime-Valid and Feature-Aware Auditing of Conditional Quantile Forecasters

Ivane Antonov, Sohom Mukherjee, Richard Pibernik, Yo Joong Choe

原文摘要

Black-box conditional quantile forecasts are widely used for sequential decisions under asymmetric costs. Once deployed, such forecasters must be monitored continuously as data streams drift and regimes change; this invalidates standard, fixed-horizon backtests for calibration. Further, existing backtests do not take into account that the notion of calibration is, in fact, information-dependent: forecasts can look calibrated to an auditor with coarse information while being miscalibrated to an auditor with richer information. We develop a distribution-free and game-theoretic testing framework for continuously auditing such forecasters with dependent, potentially nonstationary observations. We first formalize notions of conditional quantile calibration relative to the contextual features available to the auditor, showing how coarser information can limit detection. We then identify detectable alternative classes and, for contextual bets linear in the features, derive finite-time detection guarantees without an i.i.d. assumption. The resulting evidence processes provide interpretable, "feature-aware" evidence for miscalibration. We empirically validate these methods on simulated and real data, finding that a popular time series forecaster (Chronos-2) is highly miscalibrated w.r.t. multiple relevant features.

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2509.19029

Clapping: Removing Per-sample Storage for Pipeline Parallel Distributed Optimization with Communication Compression

Boao Kong, Xu Huang, Yuqi Xu, Yixuan Liang, Bin Wang, Kun Yuan

原文摘要

Pipeline-parallel distributed optimization is essential for large-scale machine learning but is challenged by significant communication overhead from transmitting high-dimensional activations and gradients between workers. Existing approaches often depend on impractical unbiased gradient assumptions or incur sample-size memory overhead. This paper introduces Clapping, a Communication compression algorithm with LAzy samPling for Pipeline-parallel learnING. Clapping adopts a lazy sampling strategy that reuses data samples across steps, breaking sample-wise memory barrier and supporting convergence in few-epoch or online regimes. Clapping comprises two variants including Clapping-FC and Clapping-FU, both of which achieve convergence without unbiased assumption for compressed gradient, effectively addressing compression error propagation in multi-worker settings. Numerical experiments validate the performance of Clapping across different tasks.

math.OC · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2510.03199

Best-of-Majority: Minimax-Optimal Strategy for Pass@$k$ Inference Scaling

Qiwei Di, Kaixuan Ji, Xuheng Li, Heyang Zhao, Quanquan Gu

原文摘要

LLM inference often generates a batch of candidates for a prompt and selects one via strategies like majority voting or Best-of- N (BoN). For difficult tasks, this single-shot selection often underperforms. Consequently, evaluations commonly report Pass@$k$: the agent may submit up to $k$ responses, and only the best of them is used when computing regret. Motivated by this, we study inference scaling in the more general Pass@$k$ inference setting, and prove that neither majority voting nor BoN exhibits the desirable scaling with $k$ and the sampling budget $N$. Combining the advantages of majority voting and BoN, we propose a new inference strategy called Best-of-Majority (BoM), with a pivotal step that restricts the candidates to the responses with high frequency in the $N$ samples before selecting the top-$k$ rewards. We prove that when the sampling budget is $N=\tildeΩ(C^*)$, the regret of BoM is $O(ε_{\mathrm{opt}}+\sqrt{ε_{\mathrm{RM}}^2C^*/k})$, where $C^*$ is the coverage coefficient, $ε_{\mathrm{RM}}$ is the estimation error of the reward model, and $ε_{\mathrm{opt}}$ is the estimation error of reward at the optimal response. We further establish a matching lower bound, certifying that our algorithm is minimax optimal. Beyond optimality, BoM has a key advantage: unlike majority voting and BoN, its performance does not degrade when increasing $N$. Experimental results of inference on math problems show BoM outperforming both majority voting and BoN.

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2604.10672

One-Step Score-Based Density Ratio Estimation

Wei Chen, Qibin Zhao, John Paisley, Junmei Yang, Delu Zeng

原文摘要

Density ratio estimation (DRE) is a useful tool for quantifying discrepancies between probability distributions, but existing approaches often involve a trade-off between estimation quality and computational efficiency. Classical direct DRE methods are usually efficient at inference time, yet their performance can seriously deteriorate when the discrepancy between distributions is large. In contrast, score-based DRE methods often yield more accurate estimates in such settings, but they typically require considerable repeated function evaluations and numerical integration. We propose One-step Score-based Density Ratio Estimation (OS-DRE), a partly analytic and solver-free framework designed to combine these complementary advantages. OS-DRE decomposes the time score into spatial and temporal components, representing the latter with an analytic radial basis function (RBF) frame. This formulation converts the otherwise intractable temporal integral into a closed-form weighted sum, thereby removing the need for numerical solvers and enabling DRE with only one function evaluation. We further analyze approximation conditions for the analytic frame, and establish approximation error bounds for both finitely and infinitely smooth temporal kernels, grounding the framework in existing approximation theory. Experiments across density estimation, continual Kullback-Leibler and mutual information estimation, and near out-of-distribution detection demonstrate that OS-DRE offers a favorable balance between estimation quality and inference efficiency.

stat.ML · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2603.02429

Dimension-Independent Convergence of Underdamped Langevin Monte Carlo in KL Divergence

Shiyuan Zhang, Qiwei Di, Xuheng Li, Quanquan Gu

原文摘要

Underdamped Langevin dynamics (ULD) is a widely-used sampler for Gibbs distributions $π\propto e^{-V}$, and is often empirically effective in high dimensions. However, existing non-asymptotic convergence guarantees for discretized ULD typically scale polynomially with the ambient dimension $d$, leading to vacuous bounds when $d$ is large. The main known dimension-free result concerns the randomized midpoint discretization in Wasserstein-2 distance (Liu et al.,2023), while dimension-independent guarantees for ULD discretizations in KL divergence have remained open. We close this gap by proving the first dimension-free KL divergence bounds for discretized ULD. Our analysis refines the KL local error framework (Altschuler et al., 2025) to a dimension-free setting and yields bounds that depend on $\mathrm{tr}(\mathbf{H})$, where $\mathbf{H}$ upper bounds the Hessian of $V$, rather than on $d$. As a consequence, we obtain improved iteration complexity for underdamped Langevin Monte Carlo relative to overdamped Langevin methods in regimes where $\mathrm{tr}(\mathbf{H})\ll d$.

cs.LG · math.OC · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38243

Nearly Minimax Optimal Regret for Adversarial Linear Bandits with Strong Corruptions

Qiwei Di, Quanquan Gu

原文摘要

We study adversarial linear bandits with strong corruption, in which an adaptive adversary chooses the reward vectors and a strong adversary corrupts each observed reward after seeing the learner's action, subject to a known total corruption budget $C$. Strong corruption breaks the importance-weighted reward estimates used by adversarial bandit algorithms: by corrupting only the rounds in which a rarely sampled action is drawn, the adversary amplifies each unit of corruption by the inverse of its sampling probability. The best known bound for this problem, $\tilde O(\sqrt{d^3T} + dC)$ by Liu et al. (2024), is a factor $\sqrt{d}$ above the corruption-free rate in its leading term, and whether $\tilde O(d\sqrt{T} + dC)$ regret is achievable was left open. We propose CRW-FTRL, which runs follow-the-regularized-leader with a radius-weighted entropy regularizer and estimates rewards with a normalized kernel estimator whose sensitivity to corruption is governed by per-action radii instead of inverse sampling probabilities. The radii are nondecreasing and are chosen jointly with the sampling distribution through a monotone log-determinant barrier problem, so that a negative linear term in the regularizer cancels both the corruption bias and the estimation error of every comparator. For $K$ actions in $\mathbb{R}^d$, CRW-FTRL achieves $\tilde O(\sqrt{dT\log K} + dC)$ regret with high probability against an adaptive adversary, which matches the best known corruption-free rate and the $Ω(dC)$ lower bound up to logarithmic factors. Applied to a discretization of a general compact action set, it yields $\tilde O(d\sqrt{T} + dC)$ regret, which settles the statistical rate of corrupted adversarial linear bandits under strong corruption.

cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2607.15530

On the Role of Normalization in Binary Iterative Hard Thresholding for 1-bit Compressed Sensing

Arya Mazumdar, Prateeti Mukherjee

原文摘要

Binary Iterative Hard Thresholding (BIHT) is a simple, yet effective, greedy method for recovering a sparse vector from one-bit sign measurements. In its original form, BIHT performs a ``gradient-descent'' step, followed by hard thresholding. A convergence analysis of this algorithm was left open in the introductory work of Jacques, Laska, Boufounos, and Baraniuk [IEEE Trans. Inform. Theory (2013)] and has remained unresolved for over a decade, with subsequent sharp analyses studying a normalized variant instead, that additionally projects every iterate onto the unit sphere. This paper resolves that gap and characterizes when per-iterate normalization is algorithmically necessary. In the noiseless setting, we prove a universal, sample-optimal convergence theorem for the original BIHT algorithm. Specifically, with $\widetilde O(s/ε)$ measurements, a deterministic, computable finite-time iterate has directional error at most $ε$, simultaneously for every $s$-sparse unit vector. This matches the optimal sample dependence achieved by normalized BIHT in prior work. Under sign corruptions, we prove a sharp separation. If at most a $τ$ fraction of signs are flipped adversarially, then BIHT, without per-iterate normalization, still reaches the robust error floor at an early iterate with a matching $\widetilde O(s/ε)$ sample complexity rate as its normalized variant. This recovery, however, is not stable. We prove a scalar lower bound showing that any nontrivial corruption pattern, even one that involves only one flipped sign together with one clean sign, forces the iterates to oscillate indefinitely. Consequently, no general last-iterate convergence theorem can hold for BIHT under sign corruptions, while its normalized surrogate provably escapes this instance.

cs.IT · cs.DS · math.ST · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38212

Scalable Bayesian Quadrature via Operator-Adapted Features

Huanyan Zhu, Cheng Li

原文摘要

Kernel methods face two recurrent bottlenecks: the cubic cost of inverting an $n\times n$ kernel matrix, as in kernel ridge regression, and the evaluation of kernel integrals, as required in kernel quadrature. Existing low-rank kernel and Gaussian process approximations primarily address the former bottleneck, but do not control the downstream quadrature error when both the kernel matrix and kernel mean are approximated, particularly when the integrand lies outside the kernel reproducing kernel Hilbert space. We address both challenges through an operator-adapted finite-feature approximation. Specifically, we construct a finite-dimensional working kernel from the eigenvalues and eigenfunctions of a second-order differential operator. We establish convergence guarantees for the resulting finite-feature estimators in kernel quadrature, with extension to other related kernel methods. Our results show that, with $m \lesssim n$ basis functions, the truncated estimator attains the corresponding benchmark statistical rates. The theory covers fixed/random designs, noisy/noiseless observations, oversmoothing/undersmoothing regimes, and Matérn/Gaussian-type weights. We provide explicit error bounds that quantify the roles of the spectral cutoff, regularization parameter, lengthscale, sampling geometry, and operator boundary condition. We verify the convergence results in numerical experiments.

math.ST · math.NA · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2609.26707

Optimal Expert Annotation for Evaluating Sequential Decision Policies from Text Records

Woojin Chae, Ezinne Nwankwo, Haitong Qin, Angela Zhou

原文摘要

Offline reinforcement learning and off-policy evaluation evaluate dynamic decision rules based on historical data, without the ability to deploy online. In recent artificial intelligence (AI) applications, simple state and reward information is recorded as complex text or image, expert annotation may be available but at a higher cost. Recent AI and machine learning (ML) predictors can label data cheaply but with unknown bias. We show how a limited budget for ground-truth data-annotation can safely make use of plentiful ML predictions in a new estimator for doubly-robust off-policy evaluation with missing rewards. Our method directs where data should be collected by optimizing annotation probabilities for the best downstream estimation variance, which we implement in a feasible batch-adaptive approach. Our work is motivated by a collaboration with a homelessness services nonprofit that writes casenotes for individuals over time. Our method can be used to unlock trustworthy inference from casenote data and answer new inferential questions such as: how does expanding outreach effort over time affect progress towards a housing application and improvement in housing placement? In simulations and on two real datasets -- casenotes from the nonprofit and human-preference votes from LMArena -- we see reductions in RMSE of 37-65% for housing placement and 21-67% for progress towards a housing application at budgets of 50% of full annotation and above, and by 55-62% at every budget on LMArena under an equal-exposure policy.

stat.ME · cs.LG · stat.ML

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2610.38151

Estimating Implicit Expected Log Density Ratios

Avi Amalanshu, Chandler Squires, Yizhou Lu, Pradeep Ravikumar

原文摘要

When exact probabilities cannot be computed, \emph{density ratio estimation (DRE)} is a key subroutine in the wide range of applications where density ratios appear, from experimental design to generative modelling and reinforcement learning. In this paper, we introduce the concept of an \emph{expected log density ratio (ELDR)}: log density ratios that are integrated under some third, possibly distinct measure. We show many problems with implicit densities where DRE is naïvely used are actually ELDR estimation tasks, and prove fundamental limitations on finite-sample ELDR estimation. Based on these results, we argue that the typical DRE-based approaches are unsuitable for ELDR estimation, and provide some foundational motivation towards bespoke ELDR estimators. To demonstrate, we propose a simple modification to some DRE methods that allows samples convergent to the integral distribution to appear in training data. We show through benchmarks that this can improve ELDR estimation error.

cs.LG · cs.AI · stat.ML

查看论文
arXiv ↗PDF ↗