跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+672新增论文
+517其中 AI 相关
+80已有论文修订
+95新增 AI 导读

arXiv 最近完成扫描:2026-11-17 04:58 · 全站最近入库:2026-11-17 04:58各信源进度 →

统计口径与数据新鲜度

统计 2026-11-16 11:05 — 2026-11-17 11:05(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-17 04:58最近发现新增版本:2026-11-17 04:58库内论文最新更新时间:2026-11-14 02:59

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

24,106 已收录论文20,158 AI 相关7,049 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 806 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-17 04:57(北京时间)
新论文 · v1arXiv:2611.10996

Toward a theory of Evaluability

Kunal Talwar

原文摘要

Machine learning models, particularly large foundation models, are increasingly evaluated along multiple axes that are relevant for their usage. In addition to accuracy measures appropriate for the specific task of interest, measures capturing safety, calibration and fairness properties are often of interest. The classical approach to these problems assumes that the test set has been held out from the training process, an assumption that is increasingly harder to justify for large foundation models. In this work, we formalize a notion of evaluability without this assumption, and ask if it is feasible to evaluate black-box models in settings where our test samples may have been used during training. We study this problem under different settings, capturing different levels of access to the model. We first study the natural setting, where the evaluator only looks at the output of the model on the test set. We fully characterize the class of distributions where accurate evaluation is possible, and show tight upper and lower bounds on the sample complexity of evaluation. Our results show that black-box evaluation in this setup is feasible if and only if the distribution is close to being small support. We then consider evaluation algorithms that can query the model on additional inputs and demonstrate connections to self-correctors studied in program testing. Using these, we show that a small amount of query access can increase the power of the evaluator for natural distributions and concept classes.

cs.LG · stat.ML

版本更新 · v2arXiv:2605.05973

Towards Reliable LLM Evaluation: Correcting the Winner's Curse in Adaptive Benchmarking

Jiefu Zhang, Yang Xu, Haixiang Sun, Zihan Zhou, Tianyu Cao, Vaneet Aggarwal

原文摘要

Adaptive prompt and program search makes LLM evaluation selection-sensitive. Once benchmark items are reused inside tuning, the observed winner's score need not estimate the fresh-data performance of the full tune-then-deploy procedure. We study inference for this procedure-level target under explicit tuning budgets. We propose SIREN, a selection-aware repeated-split reporting protocol that freezes the post-search shortlist, separates splitwise selection from held-out evaluation, and uses an item-level Gaussian multiplier bootstrap for uncertainty quantification. In a fixed-shortlist regime with smooth stabilized selection, the estimator admits a first-order item-level representation, and the bootstrap yields valid simultaneous inference on a finite budget grid. This supports confidence intervals for procedure-performance curves and pre-specified equal-budget and cross-budget comparisons. Controlled simulations and MMLU-Pro tuning experiments show that winner-based reporting can be optimistic and can change deployment conclusions, while SIREN remains close to the finite-sample reporting target.

stat.ML · cs.AI · cs.LG · stat.AP

版本更新 · v2arXiv:2607.09645

Deep Gaussian Processes on Directed Acyclic Graphs

Federico L. Perlino, Oliver Hamelijnck, Adam M. Johansen, Theodoros Damoulas

原文摘要

Many real-world processes can be represented as compositions of functions along a directed acyclic graph (DAG). In causal modelling, these correspond to the underlying mechanisms; in engineering, to multiple fidelity levels; and in gene-regulatory networks, to transcription factors. These functions are partially observed across the DAG, with noisy and heterogeneously sampled measurements, posing significant challenges for reconstruction, uncertainty propagation, and inference. To tackle these challenges, we place priors over functions and naturally arrive at Deep Gaussian Processes over DAGs. We theoretically study their prior-collapse behaviour and the effect of graph topology and intermediate observations on the preservation of information. We obtain almost-sure lower bounds on the asymptotic frequency of depths at which the distinction between inputs is preserved, identify broad kernel classes for which these hold, and prove an observation by \citep{dunlop2018} on the role of input connections. We offer a structured variational approximation that retains graph dependencies, preserves compositional uncertainty, and captures the explaining-away behaviour of colliders. Finally, we empirically validate our theory and methodology at scale and under DAG misspecification. We then model a latent-collider DAG, a protein signalling network, and a multi-fidelity \mbox{heavy-ion} collision emulation task, attaining state-of-the-art performance while recovering low-fidelity contributions and yielding interpretability over the simulator hierarchy.

stat.ML · cs.LG · math.ST · stat.CO · stat.ME

版本更新 · v3arXiv:2602.00474

Persistent-Transient Policy Evaluation for Markov Chains via Minimal Peripheral Quotients

Yang Xu, Vaneet Aggarwal

原文摘要

We study fixed-policy evaluation for finite Markov chains that may be reducible and periodic. Classical evaluation methods with gain and bias decomposition are not always diagnostic: the gain records only invariant Cesàro averages, while persistent phase-dependent behavior is absorbed into the bias together with genuinely transient effects. We identify the real peripheral invariant subspace $\mathcal{K}(P)$ of the transition matrix $P$ as the source of this ambiguity. Quotienting by $\mathcal{K}(P)$ is the minimal exact quotient that removes all non-decaying modes and makes the remaining dynamics strictly stable. After choosing a gauge projection $Π$ with kernel $\mathcal{K}(P)$, the reward admits a unique decomposition $r = g_Π^\star + (I-P)v_Π^\star$, where $g_Π^\star$ is a persistent regime profile and $v_Π^\star$ is a gauge-fixed transient component. An exact comparison with classical normalized gain and bias shows that the new pair reallocates the same information into a persistent profile and a gauge-fixed representative of the transient quotient. This decomposition reconstructs finite-horizon returns, recovers statewise average reward, admits a transient-cost interpretation, and yields a stable estimator under a generative model.

stat.ML · cs.LG · math.NA

新论文 · v1arXiv:2611.10972

JUBILEE: A Benchmark Environment for Off-Policy Evaluation in Debt Collections

Diego Silva Prado, Tiago Amaral Penteado de Jesus, Bruno Caldas de Souza, Cristiano Bevitori Maffia De Oliveira

原文摘要

Off-policy evaluation (OPE) estimates the value a candidate decision policy would achieve, using only data logged by the policy already in production. In debt collections, unrecorded factors can influence both actions and outcomes. This paper presents JUBILEE, a benchmark environment built around a debt-collections simulator that generates million-loan panels with competing risks, rare recovery events, and hidden factors driving both outcomes and assignment. The evaluation includes an 80-run comparison of tabular and gradient-boosted estimators under two synthetic outcome mechanisms and a 190-run parameter grid. Independent policy simulations provide reference values with estimated Monte Carlo uncertainty for a synthetic reward score and for simulated cash. With hidden factors influencing action assignment, several tabular methods agree closely while underestimating the field-visit policy's score, an error that also appears with boosted models. Intervals from tabular doubly robust estimation usually contain the reference without hidden assignment dependence but mostly miss it when that dependence is present. Estimates from randomized decisions do not show the same large error, although their limited precision leaves policy choice uncertain. The decision consequences depend on how policy value is defined. The field-visit policy ranks last under the score and first under simulated cash, which excludes channel costs. Under cash, log-based estimators miss this best candidate in nearly every run with hidden assignment dependence in the paired study. JUBILEE supports reproducible study of these failure modes under specified collections mechanisms.

cs.LG · q-fin.RM · stat.ML

新论文 · v1arXiv:2611.10965

Bayesian transfer for offline Q-learning with multi-task Gaussian processes

Wenxing Guo

原文摘要

Offline reinforcement learning can benefit from data collected in related tasks, yet indiscriminate knowledge transfer may lead to negative transfer under task mismatch and limited target-data coverage. We propose a target-aware Bayesian transfer framework for offline Q-learning based on multi-task Gaussian processes that jointly model task-specific Q-functions. Source tasks are assessed using target predictive evidence and maximum mean discrepancy (MMD), while posterior uncertainty and behavior-supported action sets guide conservative Bellman updates and policy extraction. To account for the dependence induced by Bellman bootstrapping, we distinguish Bellman innovations from bootstrap errors and establish stability and predictive source-selection results under explicit regularity conditions. We further derive selection-adaptive posterior contraction and offline policy-performance bounds, identifying the structural conditions under which source information can be pooled to achieve minimax-optimal rates in a shared-function Gaussian sequence experiment. Simulation studies with heterogeneous source tasks demonstrate improved target Q-function estimation and reduced material negative transfer compared with indiscriminate source pooling. An application to human decision-making data further shows higher average expected immediate payoff than both target-only learning and matched-size random source selection. These results highlight the benefits of combining target-aware source selection with uncertainty-aware conservative learning for reliable and sample-efficient offline reinforcement learning.

stat.ML · cs.LG

版本更新 · v2arXiv:2605.09654

Metropolis-Adjusted Diffusion Models

Kevin H. Lam, Tyler Farghly, Christopher Williams, Jun Yang, Yee Whye Teh, Arnaud Doucet

原文摘要

Sampling from score-based diffusion models incurs bias due to both time discretisation and the approximation of the score function. A common strategy for reducing this bias is to apply corrector steps based on the unadjusted Langevin algorithm (ULA) at each noise level within a predictor-corrector framework. However, ULA is itself a \emph{biased} sampler, as it discretises a continuous diffusion process. In this work, we consider \emph{adjusted} Langevin correctors that employ Metropolis--Hastings (MH) or Barker's accept-reject steps to correct for this bias. Since the target density ratio typically required by MH-based algorithms is unavailable, we propose methods that instead utilise the score function to compute the correct acceptance probability. We introduce the first exact method for adjusting Langevin corrections in diffusion models, based on a two-coin Bernoulli factory algorithm. We also propose an efficient approximation based on Simpson's rule that achieves accuracy of order $5/2$ in the step size at near-zero marginal cost. We demonstrate that these procedures improve sample quality on both synthetic and image datasets, yielding consistent gains in Fréchet Inception Distance (FID) on the latter.

stat.ML · cs.LG · stat.CO

版本更新 · v2arXiv:2605.01136

Spectral Graph Sparsification Preserves Representation Geometry in Graph Neural Networks

Sanjukta Krishnagopal

原文摘要

Spectral graph sparsification is a classical tool for reducing graph complexity while preserving Laplacian quadratic forms. In graph neural networks (GNNs), sparsification is often used to accelerate computation while maintaining predictive performance. In this work, we study a complementary representation-level question: does sparsification preserve the geometry of learned embeddings? For polynomial-filter GNNs, we prove that any $ε$-spectral sparsifier induces $O(ε)$ perturbations in polynomial graph filters, multilayer hidden representations, and their Gram matrices. These guarantees imply stability of squared pairwise distances, class means, and covariance structure in embedding space. We further establish finite-time training stability: under smoothness and boundedness assumptions, gradient descent on dense and sparsified graphs produces weight trajectories whose separation grows at most proportionally to the sparsification distortion. Empirically, effective-resistance sparsification validates the predicted perturbation chain on synthetic graphs and preserves hidden representation geometry on real datasets. In our experiments, the gram matrix and training dynamics show low divergence even under substantial sparsification, consistent with the predicted stability under spectral sparsification. Hidden Gram preservation strongly predicts neighborhood preservation and class-centroid stability across FashionMNIST, Cora, and Paul15. Together, these results show that spectral sparsification preserves not only graph operators, but also the representation geometry that supports downstream use of GNN embeddings for interpretability.

cs.LG · cs.SI · math.SP · stat.ML

新论文 · v1arXiv:2611.10923

Prescriptive Learning of Linear Policies with Feasibility Guarantees for Binary Decision-Making

Farzaneh Pourahmadi, Trine Krogh Boomsma, Pierre Pinson

原文摘要

Binary decision-making under uncertainty arises across a wide range of applications, including energy systems scheduling, transportation, and workforce planning. While machine learning has been increasingly applied to accelerate or replace mixed-integer solvers in such settings, existing approaches rely on predictive learning, training models to reproduce optimal decisions from labeled data, which lacks knowledge of counterfactual outcomes and provides no guarantees that the resulting decisions will be feasible. This limitation is further compounded under distributional shift, where policies trained on historical data may produce systematically infeasible decisions. We propose a prescriptive learning framework for binary decision-making that addresses both limitations. Rather than predicting optimal labels, the framework jointly optimizes over linear decision policies and feasible assignments across all training scenarios, allowing feasible alternatives and their operational performance to be considered directly during training. We further extend the framework to a distributionally robust chance-constrained formulation using a Wasserstein ambiguity set, which enforces feasibility with a prescribed probability under the worst-case distribution in the ambiguity set and enables a tunable trade-off between robustness and solution quality. We demonstrate the framework on two case studies, namely a unit commitment problem and a staff allocation problem. Out-of-sample experiments under test distributions that either match or differ from the training distribution show that the robust model is more conservative when the two distributions coincide, while under distributional shift it substantially reduces unmet demand and lowers total cost by up to 76.5% relative to classical predictive learning.

cs.LG · stat.ML

新论文 · v1arXiv:2611.10895

Robust Satisficing Ensemble: Scalable Model Aggregation Under Distribution Shifts

Ahmet Faruk Cetinkaya, Enes Agirman, Cem Tekin

原文摘要

Standard ensemble methods can improve robustness by mitigating individual model stochasticity, but they frequently fail to generalize beyond the training distribution, leaving systems vulnerable to out-of-distribution (OOD) data. We introduce the Robust Satisficing Ensemble (RSE), a scalable probabilistic aggregation framework designed to secure model combinations by minimizing their fragility against distributional changes. By bridging a robust satisficing objective with a quadratic risk surrogate, RSE derives an analytical inner adversary. This fundamental reformulation enables highly efficient weight updates without the computational bottleneck of standard minimax optimization, crucially decoupling optimization complexity from sample size to scale effectively to large datasets. In particular, we prove a geometrically decaying truncation error for the fragility approximation via Lagrange-Burmann expansion and establish stationarity guarantees for RSE. We evaluate RSE across a comprehensive spectrum of distribution shifts, including adversarial label shift (SST-5, TREC), natural subpopulation and temporal shifts (CivilComments, HuffPost). Our results demonstrate that RSE consistently outperforms strong robust baselines, including GroupDRO, IRM, and SRM. These findings establish RSE as a practical, theoretically grounded safeguard for reliable model aggregation in dynamic environments.

cs.LG · cs.AI · stat.ML

版本更新 · v2arXiv:2605.31504

When Are Multimodal Predictions Biologically Supported? A Diagnostic Evaluation Framework

Dylan Steiner, Gustavo Arango-Argoty, Gerald Sun, Etai Jacob

原文摘要

Multimodal models in oncology can produce accurate predictions, but accurate prediction does not reveal whether the model has learned biology that is shared across modalities, biology confined to one modality, or spurious correlations that reflect confounders rather than genuine biology. We introduce DECAT, a model-agnostic post-hoc evaluation framework that classifies multimodal representations into four diagnostic scenarios (shared, spurious, modality-specific, or no signal) for a given task and modality, using five null-referenced metrics and a rule-based decision procedure. The framework operates on learned representations, requires no knowledge of which specific confounder is present, and returns indeterminate when the evidence is insufficient. We evaluate multimodal architecture properties using DECAT on synthetic data across four multimodal model classes spanning over 2,500 trained representations, and demonstrate its operational use on 8,979 TCGA patients with both multimodal model embeddings and five pretrained pathology foundation models. Entangled models (e.g., CLIP) detect shared signal near-perfectly on synthetic data but also falsely claim it when it is absent, doing so in the majority of such cases on real foundation model embeddings. This false claim rate increases with signal strength and sample size, so that larger cohorts and stronger representations produce more confident but still incorrect diagnoses. Applied to both multimodal TCGA embeddings and five pathology foundation models without paired RNA, DECAT detects confounding invisible to AUROC without requiring the confounder labels, as confirmed by post-hoc stratification.

cs.LG · stat.ML

版本更新 · v4arXiv:2506.15643

Understanding Randomization in Greedy Model Search

Xin Chen, Jason M. Klusowski, Yan Shuo Tan, Chang Yu

原文摘要

We study feature subsampling in greedy model search, a randomization mechanism motivated by random forests and other ensemble methods. Although recent theory draws an analogy between this randomization and ridge regularization, these results largely rely on base learners optimized via ordinary least squares (OLS). We therefore investigate the effects of feature subsampling on greedy forward selection, a tractable abstraction of the adaptive split search used by decision trees. Assuming an orthogonal design, we prove that, under suitable conditions, ensembling with a fixed feature subsampling size can reduce expected training error without increasing degrees of freedom, thereby reducing fixed design prediction error. We also show that both the training error and degrees of freedom need not be monotone in the subsampling rate, breaking the analogy with standard shrinkage methods like the lasso or ridge regression under orthogonality. Furthermore, we characterize the exact asymptotic behavior of the estimator, showing that it adaptively reweights OLS coefficients based on their rank, with weights that are bounded by explicit logistic functions.

stat.ML · cs.LG

版本更新 · v2arXiv:2608.26552

Hadamard Flattening and Gaussian Pooling Sketch for Least Squares with Coordinate-wise Guarantee

Zhao Song, Lichen Zhang

原文摘要

Randomized sketch-and-solve algorithms accelerate overconstrained $\ell_2$ regression by replacing the input with a much smaller, randomly projected problem. While standard subspace embeddings nearly preserve the regression cost, coordinate-wise accuracy is more delicate: rather than the objective value, we want the solution vector itself to be close to the optimum in $\ell_\infty$ norm. Specifically, we seek $x'\in\mathbb{R}^d$ with $\|x'-x^\star\|_\infty\leq\fracε{\sqrt d}\|Ax^\star-b\|_2\|A^\dagger\|_{\mathrm{op}}$, where $A\in\mathbb{R}^{n\times d}$ is the design matrix, $b\in\mathbb{R}^n$ the label vector, $x^\star$ the optimal solution and $A^\dagger$ the pseudo-inverse of $A$. Price, Song and Woodruff introduced this problem and claimed that the subsampled randomized Hadamard transform (SRHT) with $O(ε^{-2}d^{1+Θ(\sqrt{\log\log n/\log d})})$ rows achieves this guarantee; we point out a gap in the proof. Song, Ye, Yin and Zhang later claimed $O(ε^{-2}d\log^3 n)$ rows, but their proof relies on an independence assumption that does not hold in general, and we exhibit an explicit instance on which it fails. To achieve a truly nearly-linear-in-$d$ row count, we introduce a new fast, dense randomized transform combining a randomized Hadamard flattening, a random permutation, and balanced, disjoint Gaussian pooling. Conditioned on the Hadamard-and-permutation stage, the sketched problem becomes an exact Gaussian regression whose noise is independent of the entire sketched design, exactly what Song, Ye, Yin and Zhang's argument was missing. Our sketch yields the $\ell_\infty$ guarantee with $m=O(ε^{-2}d\log d)$ rows, using one Hadamard pass with padded internal dimension $N=\widetilde{O}(n+ε^{-2}d^3)$. It is also efficient to apply: $(SA,Sb)$ can be computed in $O(Nd\log N)=\widetilde{O}(nd+ε^{-2}d^4)$ time.

cs.DS · cs.LG · stat.ML

版本更新 · v3arXiv:2604.19530

Calibrating Scientific Foundation Models with Inference-Time Stochastic Attention

Akash Yadav, Taiwo A. Adebiyi, Ruda Zhang

原文摘要

Transformer-based scientific foundation models are increasingly deployed in high-stakes settings, but current architectures give deterministic outputs and provide limited support for calibrated predictive uncertainty. We propose Stochastic Attention, a lightweight sample-average inference-time modification that randomizes attention by replacing softmax weights with normalized multinomial samples controlled by a single concentration parameter, and produces predictive ensembles without retraining. To set this parameter, we introduce a calibration objective that matches the stochastic attention output with the target, yielding an efficient univariate post-hoc tuning problem. We evaluate this mechanism on scientific foundation models for weather and time-series forecasting, as well as a regression task. Across benchmarks against baseline methods, Stochastic Attention achieves the strongest native calibration and the sharpest prediction intervals at comparable calibration, with adaptation costs nearly three orders of magnitude lower than the next-best baseline. Code is available at https://github.com/UQUH/StochasticAttention.

cs.LG · cs.CE · stat.ML

版本更新 · v3arXiv:2410.23244

Very fast Bayesian Additive Regression Trees on GPU

Giacomo Petrillo

原文摘要

Bayesian Additive Regression Trees (BART) is a nonparametric Bayesian regression technique based on an ensemble of decision trees. It is part of the toolbox of many statisticians. The overall statistical quality of the regression is typically higher than other generic alternatives, and it requires less manual tuning, making it a good default choice. However, it is a niche method compared to natural competitors such as XGBoost, due to the longer running time, making sample sizes above 10000-100000 a nuisance. We present a GPU implementation of BART, up to 500x faster than the previous state of the art, making BART fast enough to be a viable option on large datasets, and showing that BART parallelizes better than its non-Bayesian counterparts. We also find that, to be as practical as tree boosting on large datasets, BART would need a scalable way to infer the number of trees, which we do not provide. Our implementation is available in the Python package bartz.

stat.ML · cs.LG

新论文 · v1arXiv:2611.10803

On Generalization in Bilevel Least Squares with Overparameterized Neural Networks

Fares El Khoury, Edouard Pauwels, Samuel Vaiter, Michael Arbel

原文摘要

Bilevel optimization addresses machine learning problems in which the outer objective is a value function implicitly defined through the solution of an inner problem. While its algorithmic aspects are well developed, its generalization properties remain much less understood. In this paper, we study the generalization of inexact gradient methods when both objectives are least-squares regression problems and when the estimator of the value function's gradient, or hypergradient, is obtained by implicit differentiation in function space. This approach allows us to derive stationarity bounds for these methods in terms of the uniform generalization errors of the two regression functions needed to estimate the hypergradient. When these regression functions are defined as overparameterized two-layer ReLU networks trained by gradient flow with early stopping, we obtain explicit finite-sample bounds on these generalization errors in the Neural Tangent Kernel (NTK) regime. Combining these with our general inexact gradient analysis shows the effect of sample size on the overall convergence of the bilevel method. Numerical experiments on synthetic data illustrate our results. Beyond the NTK regime, further experiments reveal an initialization-dependent implicit bias induced by the bilevel structure, which can lead to feature learning at the inner level and overall performance improvements.

stat.ML · cs.LG

新论文 · v1arXiv:2611.10787

Trustworthy Recursive Self-Improvement with Statistical Guarantees

Zihe Gao, Zhengling Qi, Yifan Cui

原文摘要

Recursive self-improvement offers a promising path toward AI agents that improve through experience by refining their own harnesses without retraining their underlying large language models (LLMs). Making this self-improvement process reliable, however, requires more than generating new updates: agents must decide which changes are worth keeping. A greedy gate that accepts updates based solely on higher validation scores can mistake an observed gain for genuine progress and lead to performance degeneration, whereas overly conservative statistical gates may reject beneficial updates and hinder performance improvement. We introduce Trustworthy Recursive Updates with Statistical Testing (TRUST), a framework that adapts an online multiple hypothesis testing procedure to control the rate of false evolution across rounds, with an LLM analyzing historical context to help determine how much evidence is required to update the incumbent. Under interpretable assumptions, we prove that TRUST limits the proportion of accepted updates that result in unacceptable performance loss. We further derive a lower bound on the final performance gain. Across the benchmarks HotpotQA, DROP, and APPS with the models DeepSeek-V4-Flash and Qwen-3.8-Flash, TRUST reduces the average ratio of accepted inferior updates from the greedy gate's 45.59% to 2.78%, while achieving higher final scores than the state-of-the-art statistical gate baselines for recursive self-improvement. These findings demonstrate that TRUST strikes a balance between statistical reliability and performance gain in recursive self-improvement.

stat.ML · cs.AI · cs.LG · stat.ME

版本更新 · v3arXiv:2606.14095

Lyapunov-Based Sample Complexity Analysis for Weakly-Coupled MDPs

Tianhao Wu, Matthew Zurek, Weina Wang, Qiaomin Xie

原文摘要

We study the sample complexity of learning in average-reward weakly-coupled Markov decision processes (WCMDPs) and Restless Bandits (RBs) under a generative model. Naive reduction to a tabular MDP leads to high complexity bounds as the state-action space is exponentially large in the number of arms $N$. By exploiting the weakly coupled structure, we show that near-optimal policies can be learned with sample and computational complexities that are polynomial in $N$. Specifically, we analyze the plug-in approach, which applies an efficient planning algorithm to an empirical model estimated from data. For fully heterogeneous WCMDPs, we establish the first finite-sample PAC guarantee with polynomial complexity and an $O(1/\sqrt{N})$ optimality gap. For homogeneous RBs, we further prove that a smaller optimality gap is achievable under mild structural assumptions. A primary technical contribution is a novel Lyapunov-based analysis framework. Unlike classical approaches that rely on the difficult-to-control bias function, our framework uses an explicitly constructed Lyapunov function along with a drift transfer technique between the true and empirical models. A key step of independent interest in our framework is a fine-grained perturbation analysis for the underlying linear programming (LP) relaxation, which provides a general tool for analyzing LP-based policies and weakly-coupled systems.

cs.LG · math.OC · math.PR · stat.ML

新论文 · v1arXiv:2611.10738

Matchability-Aware Conformal Prediction for Open-Ended Language Model Generation

Ilia Azizi

原文摘要

Large language models (LLMs) often do not produce calibrated uncertainty estimates for their answers. We propose CAELUM, a framework that builds semantic prediction sets while separating aleatoric from epistemic uncertainty. CAELUM samples answers using an ensemble of clarification prompts, clusters them by semantic equivalence, and estimates the aleatoric and epistemic components via kernel-based von Neumann entropy before calibrating nonconformity scores with split conformal prediction. We use a weight $w_λ(x)=1+U_{\mathrm{ale}}(x)+λ\,U_{\mathrm{epi}}(x)$ to control for the contribution of the epistemic uncertainty of each instance and fit a coefficient $λ^\star$ to minimize prediction set size. When the objective is nonflat and selection is stable across splits, $λ^\star$ could be a diagnostic, where large values may indicate epistemic dominance (retrieve, abstain) and $λ^\star\to 0$ aleatoric dominance (clarify). For individual queries, a log ratio $R(x)$ compares the weighted epistemic component with the aleatoric component. Theoretically, we demonstrate that the coverage is capped by matchability, the fraction of items whose reference answer appears in any sampled cluster. Empirically, this matchability can be predicted from the original question features using a lightweight classifier. We use those predictions to define a stratum and derive a conformal coverage bound within it. Across 30 LLM and dataset combinations (4-32B), mean coverage is 0.74 in the learned stratum versus 0.65 overall, averaged over the 78 evaluations using the learned classifier. Project page: https://unco3892.github.io/caelum/

stat.ML · cs.CL · cs.LG · stat.ME

版本更新 · v2arXiv:2610.32740

Strongly Adaptive Online Learning with Time-Varying Movement Costs

Andrew Jacobsen, Emmanuel Esposito, Hao Qiu, Mengxiao Zhang

原文摘要

We study online convex optimization (OCO) with time-varying movement costs, where the learner incurs both a loss and a time-dependent penalty for changing its decisions. While previous work has established dynamic regret guarantees for this setting, obtaining strongly adaptive guarantees that hold simultaneously over arbitrary intervals remains an open challenge. We address this by developing a new algorithm which, for every interval $[a,b]\subseteq[T]$ and arbitrary comparator sequence, achieves a data-dependent regret bound of $ \widetilde{\mathcal{O}}\left( \sqrt{(D^2+DP_{[a,b]})\sum_{t \in [a,b]}(\|g_t\|^2+λ_t\|g_t\|)} \right) $, where $D$ is the diameter of the decision domain, $P_{[a,b]}$ is the path length of the comparator sequence over $[a,b]$, $g_t$ is the gradient of the loss, and $λ_t$ is the movement-cost coefficient at round~$t$. Key to our approach is a novel reduction to sleeping experts with movement costs, a previously unexplored variant that is of independent interest. As a result of our development, we further obtain strongly-adaptive dynamic regret guarantees under arbitrary delayed feedback with a refined dependence on interval-restricted cumulative delays. Our approach also naturally extends to OCO with time-varying memory, where we obtain strongly-adaptive guarantees which improve the best-known bounds.

cs.LG · stat.ML

版本更新 · v2arXiv:2605.10163

Coarsening Linear Non-Gaussian Causal Models with Cycles

Francisco Madaleno, Francisco C. Pereira, Alex Markham

原文摘要

Recent work on causal abstraction, in particular graphical approaches focusing on causal structure between clusters of variables, aims to summarize a high-dimensional causal structure in terms of a low-dimensional one. Existing methods for learning such summaries from data assume that both structures are acyclic. Acyclicity is helpful for causal effect identification and reasoning, but it excludes the many systems of scientific interest that contain feedback loops. We show that in the linear non-Gaussian (LiNG) setting, the acyclicity assumption on the high-dimensional structure can be relaxed while still allowing recovery of a low-dimensional causal directed acyclic graph (DAG). LiNG models with cycles are observationally identifiable only up to an equivalence class whose members differ only within their feedback loops. Our low-dimensional DAG, which is invariant across all members of this class, is therefore identifiable from observational data alone. While existing approaches for learning this equivalence class have exponential time complexity, our low-dimensional summary is learned in worst-case cubic time and comes with finite-sample guarantees. We provide open source code and experiments on synthetic data to corroborate our theoretical results.

stat.ML · cs.AI · cs.LG

新论文 · v1arXiv:2611.10691

A Geometric Theory for Nonequilibrium Transformer Dynamics

Aythami Bethencourt de León, Darryl D. Holm

原文摘要

We develop a variational and geometric theory based on statistical physics for the joint evolution of Transformer attention and token representations at fixed model parameters. To this end, we introduce a free energy for the attention and model the evolution of the attention weights by its Fisher--Rao gradient flow. At fixed token representations, we prove that the softmax matrix is the unique free-energy minimiser and that every positive attention trajectory converges to it. For prescribed time-dependent representations, we show that the attention dynamics becomes linear after a change to logit variables. This result yields tracking bounds relative to the instantaneous softmax and sufficient conditions for attention to converge to a fixed matrix. Hamiltonian dynamics for the representations and their momenta are then coupled to the attention relaxation driven by the same energy. This coupling results in an energy-dissipation law. We also develop two alternative Hamiltonians incorporating the attention value map. We prove global well-posedness of the attention flow and all three coupled systems.

cs.LG · math-ph · math.PR · nlin.AO · stat.ML

版本更新 · v3arXiv:2512.23818

Energy-Tweedie: Score meets Score, Energy meets Energy

Andrej Leban

原文摘要

Denoising and score estimation are classically linked through Tweedie's formula, which relates the posterior mean under Gaussian noise to the Stein score of the noisy marginal. In this work, we extend this perspective beyond Gaussian noise to a broad class of Gibbs (energy-based) noise distributions, with the generalized Gaussian family as an example. We derive the Energy-Tweedie identity: when the denoising posterior is viewed through the lens of scoring rules, the path derivative of a kernel scoring rule defined by the noise potential recovers the Stein score of the noisy marginal. Thus, the familiar correspondence between Gaussian noise, posterior means, squared loss, and Tweedie's formula is lifted to a distributional correspondence between Gibbs noise distributions, full posterior laws, kernel scoring rules, and the Energy-Tweedie identity, which gives a Tweedie-style relation for each noise potential. Among its consequences, this identity gives a posterior-samples-to-score route to score estimation, yields a principled criterion for estimating unknown noise parameters, and enables diffusion-style sampling along user-chosen paths through the noise-parameter space, supplying the score-based perspective on recent generative methods trained with scoring rules.

stat.ML · cs.LG

版本更新 · v2arXiv:2606.31963

Signed-Permutation Coordinate Transport for RMSNorm Transformers

John Sweeney

原文摘要

Modern LLM workflows move coordinate-indexed objects across checkpoints: steering vectors, sparse autoencoders, top-$k$ neuron sets, attribution lists, LoRA updates, optimizer states, and merge alignments. This requires fixing the model's residual-stream gauge. We show that the native coordinate-preserving orthogonal gauge is architecture-dependent: LayerNorm residual charts have permutation gauge $S_d$ up to a global sign flip, while RMSNorm charts with generic gains have signed-permutation gauge $B_d = S_d \ltimes \{\pm 1\}^d$. Permutation-only alignment is thus symmetry-incomplete for RMSNorm models. We introduce sign-marginalized Hungarian matching and prove that with decorrelated coordinates, signed-correlation matching has a permutation-accuracy ceiling equal to the true gauge's positive-sign fraction up to $O(d 2^{-d})$, which sign-marginalized matching removes. Omitting signs induces expected squared error $2\|ΔW\|_F^2$ on weight updates and loses orientation on vectors and AdamW first moments, while RMSNorm gains and AdamW second moments permute without signs. Controlled tests confirm these laws: a Qwen2.5-1.5B SST-2 LoRA adapter retains 0.944 accuracy (from 0.945) under $B_d$ transport vs. 0.523 under $S_d$, and steering preserves effects to $<10^{-4}$ at zero raw top-10 overlap. Across natural checkpoints, inherited slots suffice for same-base descendants (96.2%-100% retention), whereas cross-generation pairs support sparse prompt-stable subsets with causally decisive negative signs ($+0.185$ vs. $-0.010$ at $8σ$ on Qwen2/2.5-0.5B; $+0.189$ vs. $-0.021$ on 1.5B). Along fine-tuning trajectories, composing local $B_d$ gauges recovers 91.1% of cross-run Qwen2.5-1.5B coordinates at 1,500 steps vs. 60.3% for endpoint matching. Merge and interpretability audits confirm that coordinate names are reproducible only relative to an explicit $B_d$ gauge.

cs.LG · cs.CL · stat.ML

版本更新 · v2arXiv:2605.10206

Extended Wasserstein-GAN Approach to Causal Distribution Learning: Density-Free Estimation and Minimax Optimality

Shu Tamano, Masaaki Imaizumi

原文摘要

Distributional causal inference requires estimating not only average treatment effects but also interventional outcome distributions, including quantiles, tail risks, and policy-dependent uncertainty. As a method for distributional causal inference, generative adversarial network (GAN)-based counterfactual methods are flexible tools for this task. However, these methods have several limitations. First, the objectives of certain techniques do not coincide with the statistical risk of the identifiable causal target, and therefore provide limited theoretical guarantees regarding estimable counterfactual distributions or optimality. Second, implementing target-design risks under observational sampling can require density-ratio estimation. In this paper, we propose GANICE (GAN for Interventional Conditional Estimation) with several advantages: it (i) specifies the conditional interventional distribution for each treatment--covariate state as the causal estimation target; (ii) estimates the conditional distribution such that its averaged Wasserstein risk is minimized; (iii) attains minimax optimality. GANICE achieves these advantages through the extended Wasserstein distance and cell-normalized estimation, with minimax bounds based on Wasserstein GAN oracle inequalities and an anisotropic Assouad construction; Besov theory characterizes the cellwise critics. Our experiments show that GANICE outperforms existing methods on most benchmarks.

math.ST · cs.LG · stat.ML

版本更新 · v2arXiv:2607.23182

Beyond ICA: Identifiability by Symmetry Breaking

Pengzhou Wu

原文摘要

We prove the population-level identifiability of deep generative models (DGMs) with piecewise-affine (PWA) decoders and Gaussian mixture model (GMM) priors, in a purely unsupervised setting. We introduce three algebraic contrast principles for symmetry breaking: *domain contrast*, which trivializes the mixture symmetry group, a label-free counterpart of the sufficient-variability assumption of nonlinear ICA; *mechanism contrast*, which ensures every decoder branch gets an observational boundary across which it toggles alone; and *interaction contrast*, which forbids parameter conspiracies between latent components and decoder branches. Together they exploit the interplay between the discrete combinatorics of the PWA map and the continuous symmetry structure of the latent GMM. Continuity is replaced by algebraic symmetry conditions; injectivity is decoupled from structural identification and required only for pointwise inversion. Our results form a hierarchy: from *law identifiability* (LID; latent distribution up to a global affine map) through *map identifiability* (MID; decoder up to the same map) to *posterior* and *pointwise identifiability*. The ICA-form ambiguity emerges under conditions on diagonal component covariances. The interaction contrast is the only one of our contrasts required on the estimator side. To our knowledge this is the first to develop algebraic symmetry-breaking into an identifiability engine for a concrete nonlinear model class, the first to admit discontinuous decoders, and the first to handle fully non-injective decoders, where every observation admits multiple latent codes.

stat.ML · cs.LG · math.ST

版本更新 · v2arXiv:2606.09257

BSTabDiff: Block-Subunit Diffusion Priors for High-Dimensional Tabular Data Generation

Al Zadid Sultan Bin Habib, Md Younus Ahamed, Prashnna Gyawali, Gianfranco Doretto, Donald A. Adjeroh

原文摘要

High-Dimensional Low-Sample Size (HDLSS) tabular domains (e.g., omics) are characterized by $n \ll m$, where $n$ = number of samples, and $m$ = number of features. Such domains often exhibit strong local correlation groups, sparse cross-group dependencies, heavy-tailed non-Gaussian marginals, heteroscedastic noise, and structured missingness, making direct density learning in $\mathbb{R}^m$ ill-conditioned since $n \ll m$. We propose BSTabDiff, a block-subunit generative framework that partitions the $m$ observed features into $M$ latent blocks ($M \ll m$) and generates each block via a shared low-dimensional subunit variable, concentrating global dependence learning in the compact block-latent space $\mathbb{R}^M$ while decoding to the full feature space with copula-driven dependence, flexible per-feature marginals, and explicit missingness mechanisms. BSTabDiff can induce a data-driven feature ordering and block construction before latent partitioning, allowing the model to better align with local dependency structure when such organization is beneficial. BSTabDiff supports modern deep priors on block latents, including diffusion and normalizing flows, enabling stable synthesis and controllable benchmark generation in the HDLSS regime. Empirically, BSTabDiff produces more realistic and more stable high-dimensional synthetic data when compared with unstructured tabular generators on HDLSS data.

cs.LG · cs.AI · stat.ML

版本更新 · v3arXiv:2605.07964

Asymptotically Log-Optimal Bayes-Assisted Confidence Sequences for Bounded Means

Valentin Kilian, Stefano Cortinovis, François Caron

原文摘要

Confidence sequences based on test martingales provide time-uniform uncertainty quantification for the mean of bounded i.i.d. observations without parametric distributional assumptions. Their practical efficiency, however, depends strongly on the choice of martingale updates, and many existing constructions do not exploit prior information about plausible data-generating distributions or mean values. We propose a Bayes-assisted framework that uses a Bayesian working predictive model to adaptively construct confidence sequences. For each candidate mean and time point, the predictive distribution selects, among valid one-step martingale factors, the update maximising predictive expected log-growth; validity is therefore preserved even when the prior or working model is misspecified. We prove that if the predictive distribution is Wasserstein-consistent, the resulting procedure is asymptotically log-optimal, matching the per-sample log-growth of an oracle procedure with access to the true distribution. We instantiate the framework using robust predictives based on Dirichlet-process mixtures and Bayesian exponentially tilted empirical likelihood. Experiments on synthetic data, sequential best-arm identification for LLM evaluation, and prediction-powered inference show that informative priors can substantially reduce confidence-sequence width and sampling effort while retaining anytime-valid coverage.

stat.ML · cs.LG

版本更新 · v3arXiv:2503.03283

Exploring specialization and sensitivity of convolutional neural networks under simultaneous image augmentations

Pavel Kharyuk, Sergey Matveev, Ivan Oseledets

原文摘要

Drawing parallels with the way biological networks are studied, the authors adapt the treatment--control paradigm to explainable artificial intelligence research and enrich it through multi-parametric input alterations. A framework for investigating internal inference impacted by input data augmentations is proposed to address the real-world scenario with multiple simultaneously applied treatments instead of observing a single effect. The internal changes in network operation are reflected in activation changes measured by variance, which can be decomposed into components related to each augmentation by employing Sobol indices and Shapley values. To facilitate distinguishing model-related specifics from the self-interactions of augmentations, an auxiliary variable responsible for their permutations has been incorporated into the analysis. The resulting quantities enable one to visualize sensitivity to different variables and use them as features for further analysis and processing, including guided masking of activations. In addition, a method for single-class sensitivity analysis is introduced through a pipeline that involves class selection according to their match with the prediction bias generated by targeted damaging of the activations. Three convolutional networks, namely AlexNet, VGG11, and ResNet18, have been evaluated via the designed framework, with two vision transformers, ViT-b-16 and Swin-t, partially complementing the findings. Two sets of augmentations, consisting of five and seven distinct input transformations applied simultaneously, have been utilized as sources of variance in the image domain. Relying on the observed parallels, the authors assume that the developed framework can potentially be transferred to the study of biological neural networks in complex environments, as well as enhance the design of autonomous decision-making systems.

stat.ML · cs.AI · cs.LG · math.NA

新论文 · v1arXiv:2611.10570

Deep backward multistep high-order regression methods for high dimensional nonlinear parabolic partial differential equations

Qiang Han, Shaolin Ji, Zihao Xv

原文摘要

We propose a deep backward multistep high-order regression (MHDBR) method for semilinear parabolic partial differential equations through their backward stochastic differential equation representations. The method combines multistep time discretization with regression targets formed by conditional expectations. Under explicit regularity, starting-value, forward-approximation and network-moment assumptions, we prove an error bound separating time discretization, conditional Monte Carlo sampling and neural-network approximation. The analysis assumes exact population minimization; finite-training-sample generalization and optimization errors are not included. For generators independent of the gradient variable, suitably constrained GroupSort networks and sufficiently large conditional sample sizes give order $J$ in the stated nodal $L^2$ error norm. The required network sizes depend on dimension, so this result does not establish dimension-independent complexity. We present a reproducible time-discretization test satisfying the analytical assumptions and three broader neural-network benchmarks, whose relation to the assumptions is discussed explicitly.

stat.ML · cs.LG · math.AP