跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1112新增论文
+875其中 AI 相关
+133已有论文修订
+97新增 AI 导读

arXiv 最近完成扫描:2026-11-20 06:44 · 全站最近入库:2026-11-19 19:09各信源进度 →

统计口径与数据新鲜度

统计 2026-11-19 06:58 — 2026-11-20 06:58(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-20 06:44最近发现新增版本:2026-11-19 19:09库内论文最新更新时间:2026-11-19 02:59

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

27,982 已收录论文23,249 AI 相关8,559 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 955 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-20 06:42(北京时间)
新论文 · v1arXiv:2611.11667

Nonasymptotic causal structure learning in generalized linear DAGs with continuous optimization

Chang Deng, Inbeom Lee, Bryon Aragam

原文摘要

We study the problem of learning directed acyclic graphs from observational data in nonlinear models and provide finite-sample, nonasymptotic statistical guarantees in these models. Although widely studied, statistical guarantees for gradient-based methods in nonlinear models are missing from the literature. Under a generalized linear model for each node, we analyze a global nonsmooth objective combining an exact acyclicity constraint with a quasi-minimax concave penalty, which can be optimized via standard gradient-based solvers. We establish finite-sample guarantees under this model, including the most widely used logistic and Poisson models. Experiments corroborate structure recovery accuracy and computational performance, including both standardized data and negative controls as baselines.

stat.ME · cs.LG · math.ST · stat.ML

新论文 · v1arXiv:2611.11649

A Trust Region Approach for Learning Schrödinger Bridges

Takeshi Koshizuka, Denis Blessing, Max Zimmer, Sebastian Pokutta, Lorenz Richter

原文摘要

We study the numerical approximation of Schrödinger Bridges in the setting where samples from the target distribution are unavailable and only an unnormalized target density is given. In this regime, the widely used iterative proportional fitting (IPF) procedure (and its variants) often exhibits severe numerical instabilities, stemming from ill-conditioned underlying control problems and from the fact that the optimal solution may lie far from the initialization. To address these issues, we propose a trust region variant of IPF that constrains successive updates to remain at a fixed Kullback-Leibler divergence from the current iterate. This restriction significantly improves numerical stability. Moreover, we show that the resulting method admits an interpretation as a mirror descent scheme with an adaptively chosen step size. We demonstrate the effectiveness of our approach on several challenging sampling tasks.

cs.LG · stat.ML

版本更新 · v2arXiv:2605.14343

Nearest-Neighbor Radii under Dependent Sampling

Yuanyuan Gao, Yilong Hou, Zhexiao Lin

原文摘要

Nearest-neighbor methods are fundamental to classical and modern machine learning, yet their geometric properties are typically analyzed under independent sampling. In this paper, we study the nearest-neighbor radii under dependent sampling. We consider strong mixing dependent observations and ask whether dependence changes the scale of nearest-neighbor neighborhoods. We establish distribution-free almost sure convergence under polynomial mixing and sharp non-asymptotic moment bounds under geometric mixing. The moment bounds depend on the local intrinsic dimension rather than the ambient dimension, making the results applicable to high-dimensional data supported on lower-dimensional manifolds. Synthetic experiments and real-world time-series benchmarks illustrate the theory, showing that nearest-neighbor geometry remains informative under dependent sampling.

cs.LG · math.ST · stat.ML

版本更新 · v3arXiv:2610.10600

The optimal information complexity of VC learning

Steve Hanneke, Juexiao Wang

原文摘要

Steinke and Zakynthinou(2020) introduces the Conditional Mutual Information (CMI) framework of analyzing the information complexity of learning algorithms based on algorithm-dependent information-theoretic quantities. We study one of these quantities, the evaluated Conditional Mutual Information (eCMI). It has been an interesting question whether the optimal PAC guarantee for VC classes can be recovered from the algorithm-dependent analyses via CMI. And we show that it is possible to recover this guarantee by constructing a learning algorithm whose eCMI is of order O(d) in the realizable case, where d is the VC-dimension of the concept class. Specially, our algorithm is a randomized Majority-of-5 base learners with optimal in-expectation generalization guarantee.

stat.ML · cs.IT · cs.LG · math.ST

版本更新 · v3arXiv:2410.04655

Graph Fourier Neural Kernels: Learning Full PDE Trajectories Across Geometries and Anisotropic Parameter Fields

Shane E. Loeffler, Zan Ahmad, Shiyi Chen, Syed Yusuf Ali, Carolyna Yamamoto, Kensuke Sakata, Dan M. Popescu, Alana Yee 等 11 位作者

原文摘要

High-fidelity simulations of time-dependent nonlinear partial differential equations (PDEs) are expensive to repeat across families of domains and parameter fields, a cost that recurs in design optimization, uncertainty quantification, and computational medicine. Most neural-operator surrogates require each domain to be mapped onto a common grid or reference shape, and few predict the full time course. We introduce Graph Fourier Neural Kernels (G-FuNK), a neural operator for second-order semilinear PDEs with a diffusive principal part that transfers across geometries and anisotropic diffusion tensors without coordinate maps or diffeomorphisms. A weighted graph and its Laplacian supply a spectral basis that encodes geometry and anisotropy. Shared layers act in this basis, at a cost linear in mesh size after one partial eigendecomposition per geometry, and a neural ODE integrator propagates the learned generator to full spatiotemporal trajectories on unseen domains. Across anisotropic heat flow, stiff reaction-diffusion, FitzHugh-Nagumo, and patient-specific left-atrial electrophysiology on CT-derived surfaces, G-FuNK reproduces held-out trajectories with relative $L^2$ error between 0.046 and 0.160. Its accuracy is comparable to that of a geometry-aware Fourier neural operator on regular domains, and its predictions are unchanged when a rectangular domain is rotated. On left atria, whose venous and valvular openings prevent mapping onto a reference shape, it predicts an 85-ms trajectory in under half a second, against about 5 minutes for finite elements, with a mean activation-time error of 1.73 ms. Accuracy depends on graph construction and is highest when the learned generator is integrated at its training time step.

cs.LG · cs.AI · math.SP · stat.ME · stat.ML

版本更新 · v2arXiv:2605.29351

Attention as In-Context Empirical Bayes: A Two-Stage View via Particle Dynamics

Matthew Smart, Soumya Ganguly, Nilava Metya, Alexandre V. Morozov, Anirvan M. Sengupta

原文摘要

We study minimal attention-only transformers under all-token corruption and show they admit a two-stage empirical Bayes interpretation. A single attention step computes a kernel-weighted posterior mean with respect to the empirical distribution defined by the context. Depth refines this distribution through particle dynamics (Stage 1), while a long-range skip-connection carries the noisy input as a query for posterior inference (Stage 2), revealing distinct statistical roles for depth and attention residuals. The framework isolates a minimal setting in which the context itself induces a depth-dependent energy landscape governing in-context inference. We show that effective denoising can emerge without an explicit noise schedule: a fixed kernel bandwidth and finite integration horizon suffice, yielding a principled depth-noise relationship. We further establish a posterior-mean recovery guarantee for Gaussian priors, showing the empirical estimator converges to the Bayes-optimal predictor under sequential asymptotic conditions. Connecting these dynamics to reverse-diffusion limits, our results provide a statistical interpretation of attention as in-context inference via sample-based posterior estimation, without explicit density modeling.

cs.LG · math.DS · stat.ML

版本更新 · v2arXiv:2607.26346

Toward a Unified Statistical Theory of Multi-View Pretraining for Neural Knowledge Graph Learning

Jifan Zhang, Miklos Racz, Suqi Liu

原文摘要

Knowledge graph learning offers a powerful framework for representing and inferring structured knowledge, with broad practical applications. Two obstacles limit current methods: labeled triples are scarce for each entity and relation, which makes expressive models hard to train, and scoring functions are designed ad hoc, which limits their generalizability and leaves them without theoretical grounding. We address both obstacles with a theoretically grounded framework, with end-to-end guarantees, that extends and subsumes existing methods. The framework proceeds in two stages: multi-view unsupervised pretraining that integrates heterogeneous sources of entity side information into a shared representation, followed by supervised learning across multiple relation types. We establish a unified nonasymptotic risk bound that combines the subspace recovery error of pretraining and the sample complexity of the labeled data in a single guarantee, quantifying formally how large-scale unlabeled data from multiple views benefits downstream knowledge prediction. Synthetic experiments validate each component of the theory, and experiments on large-scale real-world knowledge graph benchmarks confirm the effectiveness of our approach.

math.ST · cs.SI · stat.ME · stat.ML

新论文 · v1arXiv:2611.11526

A Stability Analysis of AdamW: Unstable Equilibria and Non-Convergence

Baiyu Su, Lizhang Chen, Qiang Liu

原文摘要

AdamW is ubiquitous in deep learning, yet its behavior remains poorly understood. We analyze its dynamics through the lens of dynamical systems and show that AdamW admits an implicit fixed-point objective: its fixed points coincide with the stationary points of a constrained and regularized optimization problem. However, not all of these fixed points are stable under AdamW's dynamics, and stability depends sensitively on curvature, weight decay, and momentum parameters. Even in one-dimensional settings, equilibria may be unstable, and trajectories can exhibit persistent oscillations rather than convergence to them. We further extend the analysis to higher dimensions, deriving sufficient conditions for local stability of the continuous-time dynamics, and propose a convergent modification that preserves AdamW's fixed points. These results clarify what optimization problem AdamW is associated with, when its convergence can be expected, and how its dynamics could inspire more reliable optimizers.

cs.LG · math.OC · stat.ML

版本更新 · v2arXiv:2302.12670

Personalized Pricing with Invalid Instrumental Variables: Identification, Estimation, and Policy Learning

Rui Miao, Zhengling Qi, Cong Shi, Lin Lin

原文摘要

Pricing based on individual customer characteristics is widely used to maximize sellers' revenues. We study how a firm can learn such a personalized pricing policy from historical transactions when unobserved factors affect both the prices it charged and the revenue it earned. Standard instrumental variable (IV) methods either focus on a discrete treatment space or require that the IV affect revenue only through price. This exclusion restriction is difficult to justify in pricing, where the most natural candidate IVs are price-setting levers that may also affect revenue directly. We propose PRINT (Personalized pRicing using Invalid iNsTrumental variables), a policy learning method for continuous prices that uses such candidate IVs. Relying on structural models of revenue and price, we show that the optimal pricing policy can be identified with invalid IV that shifts higher moments of historical prices, when the unobserved factors that drive customers' price sensitivity are separated from those used by the historical pricing rule. Based on this identification, which leads to conditional moment restrictions with generalized residual functions, we construct an adversarial min-max estimator that accommodates flexible machine learning models and establish regret bounds for the learned pricing policy. Simulation studies and an application to a U.S. online auto lender illustrate the method's performance. Our results give a principled way to learn personalized prices from observational data using imperfect price-setting levers, and clarify when doing so is justified.

stat.ME · cs.LG · econ.EM · stat.ML

新论文 · v1arXiv:2611.11512

Amortized Bayesian Experimental Design with In-Context Knowledge Conditioning

Zhanghu Zhao, Daolang Huang, Xinyi Wen, Samuel Kaski

原文摘要

Amortized Bayesian experimental design (BED) enables real-time design strategies by shifting acquisition cost offline, but existing methods remain tied to the prior and task distribution they were trained on and cannot exploit additional information available at deployment. We introduce in-context amortized BED with unified knowledge encoding (IMBUE), a BED framework that admits external knowledge at deployment through a unified in-context interface. IMBUE accepts two forms of such knowledge without retraining, namely user-specified prior knowledge and observations from previous instances of the experiment. Both are encoded as additional input tokens and processed in-context with the accumulated observations by a shared Transformer, and a learned reliability filter scores each external token against the observations and excludes inconsistent ones. Across four benchmarks, IMBUE accelerates early-stage information acquisition when the external knowledge is reliable, and remains close to the no-external baseline when it is not.

stat.ML · cs.LG

版本更新 · v2arXiv:2601.21104

Multilevel and Sequential Monte Carlo for Training-Free Diffusion Guidance

Aidan Gleich, Scott C. Schmidler

原文摘要

We address the problem of accurate, training-free guidance for conditional generation in trained diffusion models. Existing methods typically rely on point-estimates to approximate the posterior score, often resulting in biased approximations that fail to capture multimodality inherent to the reverse process of diffusion models. We propose a sequential Monte Carlo (SMC) framework that integrates over the full denoising distribution to construct an unbiased Monte Carlo estimator of $p_θ(y\mid x_t)$, avoiding the bias of existing training-free guidance methods. To ensure computational tractability, we incorporate variance-reduction schemes based on MultiLevel Monte Carlo (MLMC). We evaluate on three image generation tasks. On CIFAR-10 and ImageNet classifier guidance, our approach improves accuracy by 18 and 38 points respectively over the strongest training-free baseline. On CelebA-HQ joint-attribute guidance, three existing methods (DPS, LGD, TFG) used within our framework all outperform the closest SMC-based comparator (TDS) by 25-37 points, and improve over the strongest baseline by 2-14 points.

stat.ML · cs.LG

版本更新 · v3arXiv:2605.14260

On the Burden of Achieving Fairness in Conformal Prediction

Ziang Gao, Pengqi Liu, Archer Yi Yang, Mouloud Belbahri, Jesse C. Cresswell, Masoud Asgharian

原文摘要

Conformal prediction is often calibrated with a single pooled threshold, but this can hide cross-group heterogeneity in score distributions and distort group-wise coverage. We study this phenomenon through the population score distributions underlying split conformal calibration. First, we derive a conservation law and lower bound showing that pooled calibration incurs irreducible group-wise coverage distortion at a scale set by cross-group quantile heterogeneity. Second, we demonstrate that the two leading fairness definitions for conformal prediction, Equalized Coverage and Equalized Set Size, are fundamentally in tension. Third, we quantify the cost of moving between policies which treat groups separately or pool them. Experiments on synthetic and real data confirm the same bidirectional trade-off after finite-sample calibration. Our results show that, for the policy families studied here, calibration choice does not remove cross-group heterogeneity; it determines whether the resulting distortion appears in the coverage or size dimension, providing a principled lens for analyzing fairness-oriented calibration choices in practice.

stat.ML · cs.LG

新论文 · v1arXiv:2611.11493

Learning through Capacity Awareness to Minimize Hospital Readmissions

Lydia T. Liu, Kara Schechtman, Pengyi Shi, Angela Zhou

原文摘要

Predictive algorithms are increasingly used to guide hospital discharge decisions, yet these models can exhibit censoring bias for vulnerable patients due to insufficient historical data or past discharge decisions. When biased readmission risk predictors are used to inform discharges, the training data generated by discharge decisions may perpetuate similar biases when predictors are re-trained, leading to a vicious cycle. We extend a queueing-based Markov decision process for hospital discharge to incorporate periodically retrained readmission risk predictors. Our key insight is that cost-optimal, "capacity-aware" operational decision policies can naturally diversify the lengths of stay at which patient outcomes are observed, enabling consistent estimation of readmission risk without explicit fairness constraints. We prove that repeated retraining under discharge-up-to policies converges to unbiased risk estimates under well-specification. In synthetic experiments and calibrated simulations based on patient-level data from a partner hospital, we find that where commonly used threshold-on-risk policies can remain persistently biased, discharge-up-to policies reduce learning error over status quo decision policies (such as thresholding on risk) by 24%-25%. These improvements to estimation also mitigate some forms of bias, reducing readmission cost disparities (adjusted for baseline risk differences between demographic groups) by 38%-63%. Our results demonstrate how capacity-aware decision policies can improve both predictive performance and fairness by addressing censoring bias.

cs.LG · cs.CY · stat.AP · stat.ML

版本更新 · v2arXiv:2603.09257

Transductive Generalization for GNNs via Optimal Transport

MoonJeong Park, Seungbeom Lee, Kyungmin Kim, Jaeseung Heo, Seunghyuk Cho, Shouheng Li, Sangdon Park, Dongwoo Kim

原文摘要

Analyzing the generalization of graph neural networks (GNNs) on node classification is challenging: message passing induces dependencies that break the i.i.d. assumption underlying standard inductive generalization theory. While distribution-free transductive theory resolves this dependency issue, existing classical bounds are either intractable or correlate poorly with empirical generalization. To fill this gap, we propose a transductive generalization bound based on optimal transport, connecting generalization to intra-class concentration and inter-class separation in the feature space. Empirically, our bound aligns strongly with the generalization gap. Building on this representation-based approach, we analyze how feature geometry evolves under repeated message passing. This explains the non-monotonic relationship between GNN depth and generalization, providing new insights into oversmoothing research. Practically, our bound serves as a geometry-aware regularizer that corrects the harmful behavior of message passing, yielding consistent performance improvements. The code is available at https://github.com/ml-postech/Transductive-OT-Gen-Bound.

cs.LG · stat.ML

新论文 · v1arXiv:2611.11460

Identifiable alignment of unpaired representations

Stas Syrota, Albert Lopez I Serrano, Johanne Franck, Søren Hauberg

原文摘要

Aligning two models without paired data requires a distance that both share, yet existing methods use Euclidean or spherical geometry. This is either an arbitrary choice or imposed by their loss function. We show that, under a precise form of the manifold hypothesis, the pullback of the data metric gives such a distance for any two models of one data-generating process. This holds whether the models are encoders, decoders or both, and whatever their architecture, loss, prior or training data. The alignment map preserves this distance and the distribution of representations, and is identifiable if and only if the data distribution breaks every symmetry of the data manifold. We show that this symmetry breaking holds generically. Identifiability makes the alignment coupling the unique optimum of Gromov--Wasserstein and of its weighted combinations with a linear program. The linear program alone has the same optimum if almost every data point has a distinct distribution of distances. The coupling, and with it the alignment map, is recovered exactly from permuted samples, even without identifiability, and consistently from independent ones. Our method, Neural Gromov--Monge Alignment (NGMA), uses no pairs or labels and outperforms every unpaired and label-informed baseline. It relies on pullback distances to align image models with different priors and single-cell models trained on disjoint cells, where Euclidean distances fail.

stat.ML · cs.AI · cs.LG · stat.ME

版本更新 · v2arXiv:2609.30212

Anchored Extra-Proximal Methods: Optimal Higher-Order Methods for Monotone Inclusion Problems

Ruichen Jiang, TaeHo Yoon

原文摘要

We characterize, up to logarithmic factors, the optimal higher-order oracle complexity of finding approximate solutions to composite monotone inclusion problems, formed by the sum of a smooth single-valued monotone operator and a maximal monotone set-valued operator. We introduce the Anchored Extra-Proximal (AEP) framework, which combines an anchored extrapolation step with an inexact anchored proximal update satisfying a relative-error condition. This framework recovers the composite Fast Extragradient method in the first-order setting and yields natural higher-order extensions through Taylor approximations of the smooth operator at the extrapolated point. For every $p\geq 2$, assuming that the $(p-1)$th derivative of the single-valued operator is $L_p$-Lipschitz continuous and the initial distance to a solution is at most $R$, we combine AEP with a bisection line search to obtain a $p$th-order method that attains a tangent residual of at most $\varepsilon$ in $\widetilde{O}((L_p R^{p}/\varepsilon)^{\frac{2}{3p-1}})$ oracle calls. We complement this result with a worst-case lower bound of $Ω((L_p R^p/\varepsilon)^{\frac{2}{3p-1}})$ for every deterministic algorithm in the $p$th-order oracle model, without imposing any prescribed update structure. For the subclass of smooth nonexpansive fixed-point problems, we further establish a lower bound of $Ω(\min\{R/\varepsilon, {(L_p R^{p}/\varepsilon)}^{\frac{2}{3p-1}}\})$. By choosing between AEP and Halpern iteration, we match this lower bound up to logarithmic factors, thereby also settling the deterministic oracle complexity of smooth nonexpansive fixed-point problems.

math.OC · cs.LG · stat.ML

版本更新 · v3arXiv:2601.21951

Diffusion Path Samplers via Sequential Monte Carlo

James Matthew Young, Paula Cordero-Encinar, Sebastian Reich, Andrew Duncan, O. Deniz Akyildiz

原文摘要

We develop diffusion-based samplers for target distributions known up to a normalising constant. To this end, we rely on the well-known diffusion path that smoothly interpolates between a simple base distribution and the target, popularised by diffusion models. We tackle the score estimation problem by developing an efficient sequential Monte Carlo sampler that evolves auxiliary variables from conditional distributions along the path, providing principled score and density estimates for time-varying distributions. To control the variance of score estimates, we further propose practical control variate schedules that incur minimal overhead. We adapt this general framework to paths induced by the Ornstein-Uhlenbeck (OU) time-reversal process, stochastic interpolants, and diffusion annealed Langevin dynamics, outlining their trade-offs. Finally, we provide theoretical guarantees and empirically demonstrate the effectiveness of our method on several synthetic and real-world datasets. Our code is available at: https://github.com/akyildiz-group/dpsmc.

stat.ML · cs.LG · stat.CO

版本更新 · v4arXiv:2609.33239

Schrödinger--Föllmer Actor--Critic: Diffusion Policy Improvement with Finite-Sample Analysis

Yuling Jiao, Lican Kang, Jerry Zhijian Yang, Jincheng Ying

原文摘要

Diffusion policies represent multimodal action distributions, but a $Q$-weighted update does not specify how to sample from the resulting target distribution. We propose Schrödinger--Föllmer Actor--Critic (SFAC), an offline-to-online reinforcement learning (RL) method for Kullback--Leibler (KL)-regularized policy improvement through conditional diffusion. A minimax Bellman critic estimates the action-value function, which defines an exponentially tilted target policy. A Doob $h$-transform expresses this update as a correction to the reference diffusion drift. The target drift is a weighted conditional mean of Brownian-bridge labels. We fit it by weighted regression using endpoints from the current policy and pointwise action-value evaluations. In the small-update regime, the KL-regularized update follows the natural policy-gradient direction, and the Doob correction represents the same local change in the space of diffusion drifts. Under suitable conditions, we derive finite-sample bounds that separate the errors from critic estimation, weighted neural drift regression, and diffusion discretization through a reverse-KL actor bound. The average-performance bound depends explicitly on the comparison coefficients over the policy iterations. Across six D4RL offline-to-online tasks, the final scores of the $Q$-guided bridge-regression implementation exceed its offline initialization on every task and Cal-QL on five tasks. We also compare with offline RL and online-only baselines.

stat.ML · cs.LG

新论文 · v1arXiv:2611.11387

Orthogonal Updates for the Win: Towards Accelerated Adaptive Minimax Optimization

Zhiwei Zhai, Xinyu Wang, Wenjing Yan, Lei Ding, Ying-Jun Angela Zhang

原文摘要

Single-loop minimax methods are appealing for modern machine learning, but standard stochastic gradient updates can be unstable and oscillatory under tightly coupled primal-dual dynamics, making performance notoriously sensitive to stepsizes. To fundamentally improve stability, we develop an orthogonal update framework tailored to minimax optimization, which applies updates along orthogonalized directions to reduce update anisotropy. However, orthogonalization reshapes the optimization geometry and makes coordinating primal-dual stepsizes even more delicate. To overcome this challenge, we develop AdaSGDA, which equips orthogonal updates with an adaptive stepsize rule driven by accumulated gradient norms. This mechanism automatically balances coupled primal-dual progress under orthogonalization, yielding an effectively parameter-free method for nonconvex-strongly-concave minimax optimization. We prove that AdaSGDA avoids problem-dependent tuning while attaining the state-of-the-art $\mathcal{O}\left(T^{-1/4}\right)$ convergence rate. We further propose AdaSGDA-VR, which incorporates a carefully designed variance reduction scheme and achieves the faster $\mathcal{O}\left(T^{-1/3}\right)$ rate. Extensive experiments demonstrate that our methods deliver competitive performance compared to existing minimax optimization methods.

cs.LG · math.OC · stat.ML

新论文 · v1arXiv:2611.11386

What Travels Across Languages? Reading, Transfer and Erasure of Cross-Lingual Concept Subspaces in Bangla Idioms

Syed Rifat Raiyan, Yousuf Ibne Kamal Niloy

原文摘要

Cross-lingual concept analysis requires distinguishing the information that a subspace retains from the classifiers and erasers that transfer through it. We study these operations in six frozen encoders using 10,359 Bangla (Bengali) idiom senses, eight input conditions, and figurative-class and sentiment labels. Class-mean directions provide the most consistent evidence of shared coarse meaning: in post hoc analyses, they outperform random-subspace, permuted-label and covariance-matched nulls in all 12 model-label tests under balanced accuracy. A prospective internal re-split supports idiom-alone sharing in 10 of 12 tests, spanning five encoders. The original confirmatory logistic-regression family rejects both planned nulls in 4 of 12 tests, where English directions gain only a few accuracy points above the majority-class floor (3.0-7.3). Classifier transfer depends on the input map and regularization: English-trained class-mean readouts beat both zero-shot nulls in 8 of 12 discovery tests, and stronger source-selected penalties improve transfer through logistic and LEACE directions. These readouts use Bangla-validation-selected layers and unlabeled target preprocessing. Input controls show greater coarse-class sharing for explicit glosses than for idiom counterparts. Reading, however, does not imply effective erasure: English-fitted LEACE removes less than one percentage point of Bangla figurative-class accuracy, compared with 4-11 points for Bangla-fitted LEACE; INLP shows a similar asymmetry. Pattern-filter analysis, covariance-weighted input maps and a binary Gaussian overlap bound explain how these operations can diverge. The analogous multiclass ordering is an empirical diagnostic, observed in 11 of 12 cells. The results support an estimator-sensitive account of cross-lingual concept sharing and motivate evaluating reading, classifier transfer, erasure and overlap separately.

cs.CL · cs.LG · stat.ML

新论文 · v1arXiv:2611.11381

Does Predictive Performance Translate to Causal Performance? Evaluating tabular foundation models as nuisance learners for causal effect estimation

Andrew R Weckstein, Issa J Dahabreh, Marinka Zitnik, Sonia Hernández-Díaz, Sebastian Schneeweiss

原文摘要

Tabular foundation models based on prior-data fitted networks (PFNs) achieve strong predictive performance, but their value for causal inference is unclear. We evaluated whether PFNs, which predict through in-context learning, could match or outperform conventional nuisance learners in supporting accurate causal estimation and reliable uncertainty quantification (inference). Learners modeled propensity scores and outcome regressions within doubly robust estimators of average causal effects, with and without cross-fitting, across synthetic and semi-synthetic settings with varying confounding complexity and effect heterogeneity. Under simple parametric data-generating processes, the PFNs were competitive with correctly specified parametric regressions and ensembles containing them. In more complex settings, they outperformed all conventional learners evaluated, including richly specified Super Learner ensembles, for both accuracy and uncertainty quantification. As sample size increased, only the two PFN models (TabPFN and Mitra) consistently approached nominal confidence interval coverage, whereas under-coverage persisted for conventional learners. In some settings, PFNs achieved near-nominal coverage and well-calibrated standard errors even without cross-fitting. At the smallest sample size (n=200) and in the higher-dimensional semi-synthetic settings, PFNs and other flexible learners exhibited poor coverage without cross-fitting. For PFNs, this appeared to be driven by propensity score overfitting consistent with in-sample memorization of treatment labels. These findings highlight the promise of prior-data fitted networks as in-context nuisance learners for causal estimation, while underscoring the need for further evaluation in challenging settings.

stat.AP · cs.AI · cs.LG · stat.ME · stat.ML

版本更新 · v4arXiv:2512.17473

Alternating Direction Method of Multipliers for Nonlinear Matrix Decompositions

Atharva Awari, Nicolas Gillis, Arnaud Vandaele

原文摘要

We present an algorithm based on the alternating direction method of multipliers (ADMM) for solving nonlinear matrix decompositions (NMD). Given an input matrix $X \in \mathbb{R}^{m \times n}$ and a factorization rank $r \ll \min(m, n)$, NMD seeks matrices $W \in \mathbb{R}^{m \times r}$ and $H \in \mathbb{R}^{r \times n}$ such that $X \approx f(WH)$, where $f$ is an element-wise nonlinear function. We evaluate our method on several representative nonlinear models: the rectified linear unit activation $f(x) = \max(0, x)$, suitable for nonnegative sparse data approximation, the component-wise square $f(x) = x^2$, applicable to probabilistic circuit representation, and the MinMax transform $f(x) = \min(b, \max(a, x))$, relevant for recommender systems. The proposed framework flexibly supports diverse loss functions, including least squares, $\ell_1$ norm, and the Kullback-Leibler divergence, and can be readily extended to other nonlinearities and metrics. We illustrate the applicability, efficiency, and adaptability of the approach on real-world datasets, highlighting its potential for a broad range of applications.

eess.SP · cs.LG · math.OC · stat.ML

版本更新 · v3arXiv:2603.14094

Maximin Robust Bayesian Experimental Design

Hany Abdulsamad, Sahel Iqbal, Christian A. Naesseth, Takuo Matsubara, Adrien Corenflos

原文摘要

We address the brittleness of Bayesian experimental design under model misspecification by formulating the problem as a max--min game between the experimenter and an adversarial nature subject to information-theoretic constraints. We demonstrate that this approach yields a robust objective governed by Sibson's $α$-mutual information~(MI), which identifies the $α$-tilted posterior as the robust belief update and establishes the Rényi divergence as the appropriate measure of conditional information gain. To mitigate the bias and variance of nested Monte Carlo estimators needed to estimate Sibson's $α$-MI, we adopt a PAC-Bayes framework to search over stochastic design policies, yielding rigorous high-probability lower bounds on the robust expected information gain that explicitly control finite-sample error.

stat.ML · cs.LG · math.ST · stat.CO · stat.ME

版本更新 · v4arXiv:2510.12026

Mamba Can Learn Low-Dimensional Targets In-Context via Test-Time Feature Learning

Junsoo Oh, Wei Huang, Taiji Suzuki

原文摘要

Mamba, a recently proposed linear-time sequence model, has attracted significant attention for its computational efficiency and strong empirical performance. However, a rigorous theoretical understanding of its underlying mechanisms remains limited. In this work, we provide a theoretical analysis of Mamba's in-context learning (ICL) capability by focusing on tasks defined by low-dimensional nonlinear target functions. Specifically, we study in-context learning of a single-index model $y \approx g_*(\langle \boldsymbolβ, \boldsymbol{x} \rangle)$, which depends on only a single relevant direction $\boldsymbolβ$, referred to as feature. We prove that Mamba, pretrained by gradient-based methods, can achieve efficient ICL via test-time feature learning, extracting the relevant direction directly from context examples. Consequently, we establish a test-time sample complexity that improves upon linear Transformers -- analyzed to behave like kernel methods -- and is comparable to nonlinear Transformers, which have been shown to surpass the Correlational Statistical Query (CSQ) lower bound and achieve near information-theoretically optimal rate in previous works. Our analysis reveals the crucial role of the nonlinear gating mechanism in Mamba for feature extraction, highlighting it as the fundamental driver behind Mamba's ability to achieve both computational efficiency and high performance.

cs.LG · stat.ML

版本更新 · v2arXiv:2605.26373

Online Learning on Hidden-Convex Losses via Algorithmic Equivalence: Optimal Regret, Geometric Barrier, and Bandit Feedback

Anas Barakat, Andreas Kontogiannis, Vasilis Pollatos, Ioannis Panageas, Antonios Varvitsiotis

原文摘要

We study adversarial online learning with hidden-convex losses, i.e., nonconvex losses that become convex after a nonlinear reparameterization. Ghai, Lu and Hazan (2022) proved that, under geometric and smoothness assumptions, online gradient descent (OGD) on such nonconvex losses approximately simulates online mirror descent (OMD) on the underlying convex losses with a suitable regularizer, yielding $\mathcal{O}(T^{2/3})$ regret. They left open whether the optimal $Θ(\sqrt{T})$ regret from online convex optimization can be recovered in this hidden-convex setting. We answer this question affirmatively. More specifically, via a sharper discrete-time algorithmic equivalence argument, we prove that OGD achieves $\mathcal{O}(\sqrt{T})$ regret under the same assumptions, matching the optimal worst-case rate for adversarial online convex optimization. We also address another open question of Ghai, Lu and Hazan (2022) by clarifying the geometry required for this algorithmic equivalence. We replace the diagonal-Jacobian sufficient condition with a necessary-and-sufficient Hessian compatibility condition, thereby expanding the class of admissible reparameterizations. We complement our tight regret bound with a lower bound showing that OGD suffers $Ω(T)$ regret even with a stepsize of order $1/\sqrt{T}$, for a smooth Hessian incompatible reparameterization between convex domains and an adversarial sequence of hidden-convex losses. Finally, we extend our analysis to one-point bandit feedback and prove a $\mathcal{O}(T^{3/4})$ expected regret bound for bandit OGD with spherical smoothing, matching its classical rate on convex losses.

cs.LG · math.OC · stat.ML

版本更新 · v2arXiv:2602.12449

Computationally sufficient statistics for Ising models

Abhijith Jayakumar, Shreya Shukla, Marc Vuffray, Andrey Y. Lokhov, Sidhant Misra

原文摘要

Learning Gibbs distributions using only sufficient statistics has long been recognized as a computationally hard problem. On the other hand, computationally efficient algorithms for learning Gibbs distributions rely on access to full sample configurations generated from the model. For many systems of interest that arise in physical contexts, expecting a full sample to be observed is not practical, and hence it is important to look for computationally efficient methods that solve the learning problem with access to only a limited set of statistics. We examine the trade-offs between the power of computation and observation within this scenario, employing the Ising model as a paradigmatic example. We show that the couplings of the model can be learned to a desired accuracy from statistics whose order grows linearly with the $\ell_1$ width of the model and only logarithmically with the inverse of the accuracy. In particular, for a fixed accuracy, the required order is proportional to the width. This allows us to infer the model's structure, using statistics whose order grows logarithmically with the inverse of the weakest coupling, and also to learn its magnetic fields. When the width and the accuracy are fixed, the running time is polynomial in the number of variables. We also discuss a setting where prior information about structure of the model is available and show that the learning problem can be solved efficiently with even more limited observational power.

cs.LG · cond-mat.stat-mech · stat.ML

版本更新 · v2arXiv:2605.12492

Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation

Kexuan Shi, Hanxuan Li, Zeju Qiu, Yandong Wen, Simon Buchholz, Weiyang Liu

原文摘要

We introduce Pion, a spectrum-preserving optimizer for large language model (LLM) training based on orthogonal equivalence transformation. Unlike additive optimizers such as Adam and Muon, Pion updates each weight matrix through left and right orthogonal transformations, preserving its singular values throughout training. This yields an optimization mechanism that modulates the geometry of weight matrices while keeping their spectral norm fixed. We derive the Pion update rule, systematically examine its design choices, and analyze its convergence behavior along with several key properties. Empirical results show that Pion offers a stable and competitive alternative to standard optimizers for both LLM pretraining and finetuning.

cs.LG · stat.ML

版本更新 · v2arXiv:2605.12190

Information-Theoretic Generalization Bounds for Sequential Decision Making

Futoshi Futami, Masahiro Fujisawa

原文摘要

Information-theoretic generalization bounds based on the supersample construction are a central tool for algorithm-dependent generalization analysis in the batch i.i.d. setting. However, existing supersample conditional mutual information (CMI) bounds do not directly apply to sequential decision-making problems such as online learning, streaming active learning, and bandits, where data are revealed adaptively and the learner evolves along a causal trajectory. To address this limitation, we develop a sequential supersample framework that separates the learner filtration from a proof-side enlargement used for ghost-coordinate comparisons. Under a row-wise exchangeability assumption, the sequential generalization gap is controlled by sequential CMI, a sum of roundwise selector--loss information terms. We also establish a Bernstein-type refinement that yields faster rates under suitable variance conditions. The resulting bounds specialize to online learning, streaming active learning with importance weighting, and stochastic multi-armed bandits.

stat.ML · cs.LG

版本更新 · v2arXiv:2401.07174

On the (In)Compatibility between Group Fairness and Individual Fairness

Shizhou Xu, Thomas Strohmer

原文摘要

We study the compatibility between utility-optimized statistical parity and individual fairness. While individual fairness seeks to treat similar individuals similarly, optimal statistical parity aims to provide similar treatment to individuals who share relative similarity within their respective sensitive groups. Although both fairness perspectives are desirable, they often conflict in practical applications. Our goal in this work is to analyze the existence of this conflict and its potential solution. In particular, we establish sufficient (sharp) conditions for the compatibility between the optimal (post-processing) statistical parity $L^2$ learning and the ($K$-Lipschitz or $(ε,δ)$) individual fairness requirements. When a conflict arises, we first relax the optimal statistical parity solution to the Pareto frontier, representing the optimal trade-off between $L^2$ error and statistical disparity, and then examine its compatibility with individual fairness. Our analysis identifies regions along the Pareto frontier that satisfy individual fairness constraints. Additionally, we provide individual fairness guarantees for the composition of a trained model with the optimal post-processing step, enabling practitioners to assess the compatibility of post-processed models. This provides practitioners with a principled approach to achieving Pareto-optimal statistical parity while maintaining individual fairness constraints.

math.ST · cs.CY · cs.LG · stat.ML

版本更新 · v2arXiv:2602.08315

Reusable Conditional Resampling via Flow Matching for Constraint-Based Causal Discovery

Shunyu Zhao, Yanfeng Yang, Shuai Li, Kenji Fukumizu

原文摘要

Constraint-based causal discovery requires many conditional independence (CI) tests. Generative CI tests are attractive in nonlinear and high-dimensional settings because they can model complex conditional distributions, but often suffer from high per-test costs and limited reusability across query-dependent conditional resampling tasks in graph search. We propose a generative nonlinear CI testing and graph search framework for PC-style causal discovery. We introduce Langevin-RePaint for conditional imputation. Building on Langevin-RePaint and joint flow matching modeling, we propose the Flow Matching based Conditional Independence Test (FMCIT), which reformulates generative CI testing from query-specific conditional modeling into a reusable conditional resampling framework. We establish theoretical guarantees for FMCIT's type-I error control. FMCIT reuses a single trained generative model to generate randomized copies throughout graph search. We then introduce GPC-FMCIT, which embeds FMCIT into a guided PC pipeline with screening, guidance, refinement, and orientation. By using low-cost screening to shrink the search space, constructing edge-specific candidate conditioning sets from local graph structure, and invoking FMCIT only in a budget-controlled refinement stage, GPC-FMCIT reduces both the deployment cost of individual nonlinear CI queries and the overall query complexity. On synthetic nonlinear structural equation models, GPC-FMCIT improves CPDAG recovery over PC-stable baselines and requires far fewer FMCIT queries than direct FMCIT-based PC search, and on hESC gene-expression data it yields smaller graphs than PC/FisherZ in a descriptive comparison. Overall, our results suggest that, through the joint design of reusable conditional resampling and controlled graph search, generative nonlinear CI testing can become a practical component of PC-style causal discovery.

cs.LG · stat.ML