Many causal inference problems require sampling from interventional outcome distributions rather than estimating low-dimensional summaries. Existing approaches often rely on fitting generative models of the target distribution together with additional nuisance components, which can be computationally demanding for high-dimensional outcomes. We propose Interventional Maximum Mean Discrepancy Flow (IMMD-Flow), a particle-based method that instead represents the target interventional distribution through its kernel mean embedding. This kernel embedding admits a doubly robust, Neyman-orthogonal estimator with finite-sample guarantees and can be efficiently reused to sample from the interventional distribution via MMD gradient flow. IMMD-Flow accommodates both hard and soft interventions, admits theoretical guarantees for estimation and particle approximation, and extends to high-dimensional image outcomes through frozen encoders. Across image-based benchmarks, IMMD-Flow matches or improves upon state-of-the-art generative approaches while showing significant speed advantages.
Many problems in computational statistics and machine learning can be formulated as approximating a target distribution by minimizing a discrepancy over a restricted class of candidate distributions. This minimum-discrepancy viewpoint appears, for example, in numerical integration, sampling, generative modelling, and statistical inference, but these instances are often studied separately in the literature. This thesis develops the first unified treatment of these many problems through the minimization of the maximum mean discrepancy (MMD), a kernel-based discrepancy between probability measures. The central theme is that MMD is not only a statistically meaningful distance between probability distributions, but also a tractable optimization objective to design practical and efficient algorithms. This thesis summarizes the recent theoretical and empirical advancements of such minimum-MMD based algorithms.
Zonghao Chen, Aratrika Mustafi, Pierre Glaser, Anna Korba, Arthur Gretton, Bharath K. Sriperumbudur
We introduce a (de)-regularization of the Maximum Mean Discrepancy (DrMMD) and its Wasserstein gradient flow. Existing gradient flows that transport samples from source distribution to target distribution with only target samples, either lack tractable numerical implementation ($f$-divergence flows) or require strong assumptions, and modifications such as noise injection, to ensure convergence (Maximum Mean Discrepancy flows). In contrast, DrMMD flow can simultaneously (i) guarantee near-global convergence for a broad class of targets in both continuous and discrete time, and (ii) be implemented in closed form using only samples. The former is achieved by leveraging the connection between the DrMMD and the $χ^2$-divergence, while the latter comes by treating DrMMD as MMD with a de-regularized kernel. Our numerical scheme uses an adaptive de-regularization schedule throughout the flow to optimally trade off between discretization errors and deviations from the $χ^2$ regime. The potential application of the DrMMD flow is demonstrated across several numerical experiments, including a large-scale setting of training student/teacher networks.