Mamba-2 compresses all context into a fixed-size recurrent state that dominates decoding memory and bandwidth. Existing compressors quantize state values or prune state coordinates, scoring each channel independently in the native basis. We examine trained Mamba-2 states and find that energy is almost rank-one, but the output does not read the energetic directions: energy concentrates in a bias-like direction accumulated by the recurrence, while the content the output reads lives in a low-dimensional subspace that is not aligned with the state's coordinates. Even output-aware scores are computed coordinate by coordinate, so they miss the directions that determine the input--output map. Compression should therefore target the write--read subspace rather than individual coordinates. We propose \method{}, a post-training transform that instantiates balanced truncation with full controllability and observability Gramians and applies the rotation exactly via Mamba-2's scalar-identity transition. Because the useful rank varies widely across layers, we further introduce HARP, which allocates ranks per layer from the Hankel spectrum and protects retrieval-sensitive layers, improving both perplexity and retrieval over uniform budgets. Calibration takes a single forward pass, an order of magnitude faster than GHOST's released protocol, with lower peak memory. With HARP, \method{} halves the state dimension at an average WikiText-2 perplexity increase of $0.3$ points ($1.8\%$) across 130M--2.7B, versus $1.6$ ($9.3\%$) for GHOST, and the gap widens at 25\% retention.
Mingyuan Xu, Zongqi Xia, Tianxi Cai, Doudou Zhou, Nian Si
Learning a common policy from multiple source environments requires accounting for cross-source heterogeneity, and the offline setting introduces an additional challenge of limited data coverage. We formulate this problem as a distributionally robust Markov decision process under a shared linear MDP representation with source-specific transition and reward components. The uncertainty set is constructed through feature-wise mixtures of these components, yielding a $d$-rectangular structure that captures cross-source heterogeneity and admits a robust Bellman recursion. Building on this formulation, we develop Group-Robust Pessimistic Value Iteration (GR-PEVI), an offline policy-learning algorithm that combines feature-wise worst-case aggregation with a data-dependent pessimism penalty. Under a robust partial coverage condition, we establish finite-sample suboptimality guarantees for GR-PEVI and extend the analysis to clustered sources, characterizing the trade-off between the statistical gain from pooling similar sources and the approximation error induced by within-cluster heterogeneity. Simulation studies demonstrate the effectiveness of GR-PEVI, with empirical behavior consistent with the theoretical analysis. We apply GR-PEVI to a multi-source electronic health record study of multiple sclerosis treatment decisions to illustrate its use in a clinical setting.
Two compass bearings near north can have an ordinary arithmetic mean pointing south: the difficulty is that angles live on a circle, not a straight line. Using real wind observations, diffusion-MRI orientations, and geometric examples from machine learning, this article introduces circular and axial averages, directional variability, and probability models on circles and spheres. The guiding principle is to identify the geometry and symmetries of the observations before choosing a statistical method.
Statistical procedures often control expected loss at a prescribed level while leaving part of their risk budget unused. Building on the e-value representation of risk-controlled decisions and the e-closure principle for multiple testing, we formulate recertification, a general paradigm for improving procedures by exploiting computable slack in their validity certificates. A valid procedure's losses normalized by their risk budgets form a state-indexed e-value collection; conversely, decisions whose normalized losses are bounded by this collection inherit its risk guarantees. Recertification uses distributional structure to quantify slack, enlarge the certifying e-values while preserving validity, and construct less conservative decisions. We develop two applications. For conformal risk control (CRC) with a known loss family, recertification redistributes unused risk budget among the held-out points of each hypothetical augmented sample. The resulting controller is valid under every finite exchangeable law and no more conservative than CRC on any calibration sample. For convex losses on an interval of actions, repeated recertification converges geometrically to an admissible controller. For false discovery rate control with arbitrarily dependent e-values, recertification exploits unused conditional budgets in randomized e-BH. It yields a closed-form randomized procedure whose rejection set contains that of randomized e-BH for every realization of the shared randomization and whose expected number of rejections is maximal subject to the same conditional budget constraints. Recertifying with the larger e-collection of closed e-BH gives a second closed-form procedure whose rejection set contains that of the first. Experiments on CIFAR-100 hierarchical classification and multiple testing simulations demonstrate improved budget utilization, informativeness and power over the baselines.
Nicolás García Trillos, Adam Quinn Jaffe, Bodhisattva Sen
The quantity of interest in the classical Cramér-Rao theory of unbiased estimation (e.g., the Cramér-Rao lower bound, its exact attainment for exponential families, and asymptotic efficiency of maximum likelihood estimation) is the variance, which represents the instability of an estimator when its value is compared to the value for an independently-sampled data set from the same distribution. In this paper we are interested in a quantity which represents the instability of an estimator when its value is compared to the value for an infinitesimal additive perturbation of the original data set; we refer to this as the "sensitivity" of an estimator. The resulting theory of sensitivity is based on the Wasserstein geometry in the same way that the classical theory of variance is based on the Fisher-Rao (equivalently, Hellinger) geometry, and this insight allows us to determine a collection of results which are analogous to the classical case: a Wasserstein-Cramér-Rao lower bound for the sensitivity of any unbiased estimator, a characterization of models in which there exist unbiased estimators achieving the lower bound exactly, and various results on the asymptotic achievement of the lower bound. More precisely, we show that Wasserstein projection estimators (and one-step approximations thereof) are asymptotically sensitivity-efficient under suitable assumptions, and that there exist Hodges-type estimators which exhibit super-sensitivity-efficiency. We use these results to treat many statistical examples, sometimes revealing new optimality properties for existing estimators and other times revealing entirely new estimators.
Testing by betting is a powerful framework that recasts hypothesis testing as a betting game. A popular approach spreads the initial wealth over fixed bets according to a prior distribution, using a universal portfolio algorithm. For suitable priors, this approach achieves the optimal expected rejection time as the error probability goes to zero at a fixed alternative. Yet, priors with this guarantee can behave quite differently near the null. This motivates the question: Which prior should we use in testing-by-betting algorithms? We study bounded scalar payoffs and, under technical conditions, prove a sharp formula for expected rejection time, uniform over all bounded alternatives as the error probability tends to zero. The formula shows how the prior's mass around the zero bet determines the cost of adapting to an unknown alternative. We characterize the priors that attain the best possible leading constant when the error probability and alternative vary together. We also show that no valid test can uniformly improve on this cost across ranges of alternatives near the null. In particular, the uniform prior can require arbitrarily more observations than suitable priors that concentrate around the zero bet. A general inversion argument converts our results into sharp expected times to reach a prescribed confidence-sequence width. We also give a concrete prior choice and an FFT-based algorithm with almost linear computational complexity.
Nina Drobac, Margaux Br{é}g{è}re, Joseph de Vilmarest, Olivier Wintenberger
The evolution of a time series is often influenced not only by its past values but also by external covariates, giving rise to forecasting with exogenous variables. Nonlinear and delayed effects between covariates and the target series make this task particularly challenging. We propose a forecasting framework based on Ridge regression with sliding-window signature features, combining the computational efficiency of (semi-)parametric models with the flexibility of nonparametric approaches. These features capture complex dependencies between the target and the exogenous variables without relying on prior assumptions about the form of these relationships or the influence of past observations. We establish theoretical guarantees in the discrete-time setting, including universality of approximation and stationarity for the sliding-window transform. To ease implementation, we introduce a novel sequential algorithm that exploits window overlap for efficient signature updates. The method is evaluated on both synthetic and real electricity demand data. Results show that signature features effectively encode temporal and nonlinear dependencies, yielding accurate forecasts competitive with those based on expert knowledge and recent neural network approaches.
Xiaoyang Wu, Jiajun Liu, Haojie Ren, Changliang Zou
Novelty detection is an important area of research in both statistics and machine learning. In this paper, we focus on the conditional novelty detection problem, where novelties arise from the relationship between different variables. We first adopt the conformal inference framework and propose a data-enhanced localized conformal p-value, constructed by recalibrating an enhanced conditional distribution estimator. This estimator efficiently captures conditional information by incorporating both calibration and test data into its kernel estimation. We show that our p-values are valid in finite samples and can improve detection efficiency. Based on this p-value, we then develop a novel conditional novelty detection algorithm, along with a data-driven bandwidth selection method that ensures finite-sample false discovery rate control while enhancing detection power. Both simulated and real data experiments demonstrate the advantages of our proposed approach.
Chenxi Liu, Yongqiang Chen, Tongliang Liu, Mingming Gong, James Cheng, Bo Han, Kun Zhang
Revealing hidden causal variables alongside the underlying causal mechanisms is essential to the development of science. Despite the progress in the past decades, existing practice in causal discovery (CD) heavily relies on high-quality measured variables, which are usually given by human experts. In fact, the lack of well-defined high-level variables behind unstructured data has been a longstanding roadblock to a broader real-world application of CD. This procedure can naturally benefit from an automated process that can suggest potential hidden variables in the system. Interestingly, Large language models (LLMs) are trained on massive observations of the world and have demonstrated great capability in processing unstructured data. To leverage the power of LLMs, we develop a new framework termed Causal representatiOn AssistanT (COAT) that incorporates the rich world knowledge of LLMs to propose useful measured variables for CD with respect to high-value target variables on their paired unstructured data. Instead of directly inferring causality with LLMs, COAT constructs feedback from intermediate CD results to LLMs to refine the proposed variables. Given the target variable and the paired unstructured data, we first develop COAT-MB that leverages the predictivity of the proposed variables to iteratively uncover the Markov Blanket of the target variable. Built upon COAT-MB, COAT-PAG further extends to uncover a more complete causal graph, i.e., Partial Ancestral Graph, by iterating over the target variables and actively seeking new high-level variables. Moreover, the reliable CD capabilities of COAT also extend the debiased causal inference to unstructured data by discovering an adjustment set. We establish theoretical guarantees for the CD results and verify their efficiency and reliability across realistic benchmarks and real-world case studies.
We propose a lending protocol with far fewer risks than existing pool-based lending protocols (e.g., Aave, Compound). In our protocol, lenders and borrowers embody the spirit of decentralization by making competing interest-rate offers, resulting in the discovery of the prevailing interest rate through tâtonnement-like dynamics as opposed to taking rates partly determined by a central authority (e.g., Aave, Federal Reserve). We eliminate the risk of manipulation by creating loans with a fixed interest rate throughout the loan---like US Treasury bills. In particular, we further derive performance guarantees probabilities of repayment shortfall and liquidation by modeling the value of the collateral as a random walk, thus allowing participants to more efficiently manage their risk. To do so, we use characteristic functions and Fourier inversion to derive exact integral representations of the liquidation and repayment shortfall probabilities under the most general assumptions. We illustrate empirically this method by using Bitcoin as collateral and fitting its historical prices to Gaussian, Student-$t$, and symmetric stable Paretian distributions.
Causal representation learning (CRL) aims to learn low-dimensional causal latent variables from high-dimensional observations. While identifiability has been extensively studied for CRL, estimation has been less explored. We use empirical Bayes (EB) to estimate causal representations from multiple domains whose differences reflect interventions in a shared causal model. Multi-domain CRL poses a simultaneous inference problem suited to EB. Here, we propose an EB $f$-modeling algorithm that improves the quality of learned causal variables by exploiting invariant structure within and across domains. Specifically, we consider linear measurement models with interventional priors induced by an acyclic latent causal model with known graph and intervention targets. We develop an EM-style algorithm based on causally structured score matching. In experiments on synthetic and semi-synthetic single-cell data, our proposed method achieves more accurate estimation than alternative methods and baselines.
Problem definition: Subscription box services often ask customers for ratings before choosing what to send. Responses are scarce, and longer questionnaires and finer scales lower completion. We study which candidate items a questionnaire should ask about, and in what format, when the platform then selects a fixed number of them. A design is judged by the expected value lost by not selecting the truly best items. Methodology/results: We approximate each ordinal rating by a Gaussian signal carrying its information at the cutoff between selected and unselected items. As equal numbers of responses per item grow, the ratio of this proxy's loss to the exact loss tends to one. With 1 to 100 responses per item, the two are within about 7 percent. Averaging the information over the prior instead can misstate the loss by about half. For an item far from the cutoff, the value of responses grows slowly at first and then faster, so our envelope rule values responses in blocks. In simulations with a correct prior it lowers the loss by 15 to 81 percent relative to an even spread, and on MovieLens ratings by 6 to 25 percent. Under the proxy, with responses allocated optimally, formats are ranked exactly by one number, the expected information per customer asked. Choosing the format this way comes within about 2 percent of the best on average. Managerial implications: Feedback should go to items whose place in the selection is unclear. Planning helps most when the platform's prior beliefs are informative and right. A platform unsure of its prior should widen it and send the questionnaire in waves. Calibrated on MovieLens ratings, a five-point answer carries about 1.7 times the information of a like/dislike answer, so at equal length the finer scale is better when it cuts completion by less than about 40 percent.
Skyler Wu, Yash Nair, Antoine Maechler, Emmanuel J. Candès
Estimating a population mean is fundamental across scientific applications, but observing every outcome can be prohibitively expensive. Often, covariates and machine learning (ML) predictions are available for the full population, even when only a small subset of outcomes can be observed. How can we leverage this information to proactively select observations to label, update ML predictions as labels arrive, and construct statistically valid confidence intervals? We develop a general, without-replacement formulation of Pro-Active Inference (PAI), extending our earlier with-replacement approach and building on Active Statistical Inference. Rather than making fixed-order label-skip decisions, PAI proactively selects which observation to label next from the remaining covariate pool while accommodating online ML updates. We establish PAI's finite-sample unbiasedness and asymptotic validity, develop an improved finite-sample variance estimator, and extend PAI to superpopulation inference without Bonferroni corrections or $α$-splitting. We characterize PAI's potential efficiency gains over Batch Active Inference in a quasi-oracle setting, highlighting the role of prediction-residual imbalance. Across eight diverse datasets, PAI achieves greater sampling efficiency than Sequential Active Inference in the vast majority of tested settings with near-nominal statistical coverage, using the same ML predictor and uncertainty score. However, comparisons with simpler baselines such as Prediction-Powered Inference and uniform sampling, as well as the closely related Prediction-Powered Active Testing, reveal when additional adaptivity helps or hurts statistical efficiency. Through interpretable, theory-motivated diagnostics and frozen-ML ablations, we investigate when and why PAI gains or loses efficiency, highlighting the initial residual-estimation problem and subsequent online learning dynamics.
Antonin Schrab, Ilmun Kim, Mélisande Albert, Béatrice Laurent, Benjamin Guedj, Arthur Gretton
We propose two novel nonparametric two-sample kernel tests based on the Maximum Mean Discrepancy (MMD). First, for a fixed kernel, we construct an MMD test using either permutations or a wild bootstrap, two popular numerical procedures to determine the test threshold. We prove that this test controls the probability of type I error non-asymptotically. Hence, it can be used reliably even in settings with small sample sizes as it remains well-calibrated, which differs from previous MMD tests which only guarantee correct test level asymptotically. When the difference in densities lies in a Sobolev ball, we prove minimax optimality of our MMD test with a specific kernel depending on the smoothness parameter of the Sobolev ball. In practice, this parameter is unknown and, hence, the optimal MMD test with this particular kernel cannot be used. To overcome this issue, we construct an aggregated test, called MMDAgg, which is adaptive to the smoothness parameter. The test power is maximised over the collection of kernels used, without requiring held-out data for kernel selection (which results in a loss of test power), or arbitrary kernel choices such as the median heuristic. We prove that MMDAgg still controls the level non-asymptotically, and achieves the minimax rate over Sobolev balls, up to an iterated logarithmic term. Our guarantees are not restricted to a specific type of kernel, but hold for any product of one-dimensional translation invariant characteristic kernels. We provide a user-friendly parameter-free implementation of MMDAgg using an adaptive collection of bandwidths. We demonstrate that MMDAgg significantly outperforms alternative state-of-the-art MMD-based two-sample tests on synthetic data satisfying the Sobolev smoothness assumption, and that, on real-world image data, MMDAgg closely matches the power of tests leveraging the use of models such as neural networks.