跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1022新增论文
+813其中 AI 相关
+85已有论文修订
+95新增 AI 导读

arXiv 最近完成扫描:2026-11-17 04:58 · 全站最近入库:2026-11-17 13:07各信源进度 →

统计口径与数据新鲜度

统计 2026-11-16 14:18 — 2026-11-17 14:18(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-17 04:58最近发现新增版本:2026-11-17 13:07库内论文最新更新时间:2026-11-16 17:16

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

24,849 已收录论文20,754 AI 相关7,213 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 126 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-17 04:57(北京时间)
新论文 · v1arXiv:2611.13624

Multifidelity Operator Inference for Non-intrusive Model Reduction under Data Scarcity

Vivian Zhang, Tomoki Koike, Atticus Rex, Elizabeth Qian

原文摘要

Many-query analyses, in which a computational model is repeatedly evaluated, play an important role in uncertainty quantification and optimization for engineering design. However, many-query analyses can be prohibitively expensive when considering high-fidelity numerical models, especially those governed by partial differential equations (PDEs). Surrogate modeling replaces costly simulations with cheap approximations, enabling practical uncertainty quantification and design workflows. Machine learning (ML) approaches to surrogate modeling have gained significant traction in the development of surrogate models due to their non-intrusive nature; they do not require access to the underlying simulation code. The Operator Inference (OpInf) approach, in particular, fits reduced operators to state and corresponding time derivative data. However, high-fidelity training data is scarce due to limited computational budgets, and OpInf models trained on such scarce data often exhibit high variance, a critical liability in high-risk engineering design. We propose Multifidelity Operator Inference (MF-OpInf), which reduces this variance by supplementing scarce high-fidelity snapshots with abundant low-fidelity data drawn from coarser discretizations of the same problem. MF-OpInf handles data across varying fidelity levels by utilizing a multifidelity linear regression framework to estimate the reduced operators. Numerical experiments on model PDE problems demonstrate that MF-OpInf predicts PDE solution fields with higher accuracy compared to the high-fidelity-only OpInf with the same computational budget.

cs.LG · math.NA · stat.CO

新论文 · v1arXiv:2611.13523

Parallel Broyden Methods for Efficiently Evaluating Nonlinear State Space Models

Ian Christopher Tanoh, Scott W. Linderman

原文摘要

Nonlinear state space models are typically evaluated sequentially, limiting their ability to exploit modern parallel hardware. Recent work has shown that state space models can be evaluated in parallel by reformulating evaluation as a root-finding problem, which can be solved with a parallel form of Newton's method. However, exact Newton methods require expensive multiplication of dense Jacobian matrices, and quasi-Newton methods based on diagonal approximations, while more scalable, are often slow to converge because they neglect interactions across state dimensions. We introduce a novel quasi-Newton approach based on Broyden's method, which captures coupling terms with diagonal-plus-low-rank approximations to blocks of the Jacobian. We construct the approximations from trajectory secants, avoiding automatic differentiation while retaining the favorable scaling of diagonal approximations and remaining compatible with parallel scan through rank compression. We prove finite-step recovery of the exact trajectory under the compressed scan, and show that convergence is tracked by the compression error and by the block approximation error along the current trajectory error, rather than in spectral norm. Empirically, we show that our method is advantageous when interactions are approximately low rank or when Jacobian evaluation is expensive: on low-rank RNNs, parallel MCMC algorithms, and denoising diffusion models, it converges in fewer iterations than other quasi-Newton methods and is the fastest parallel method when Jacobian evaluation is expensive or state dimensions are large.

cs.LG · math.NA

版本更新 · v2arXiv:2605.09288

MC$^2$: Monte Carlo Correction for Fast Elliptic PDE Solving

Ethan Hsu, Hong Meng Yam, Ivan Ge

原文摘要

Partial differential equation (PDE) solvers underpin modern computer graphics, geometry processing, and scientific computing. Classical Monte Carlo PDE solvers such as Walk-on-Spheres (WoS) provide mesh-free estimations that are unbiased, but with accuracy limited by sampling and numerical approximation. Learned solvers are fast but biased and brittle under distribution shift. We therefore present MC$^2$, a hybrid Neural Network-Monte Carlo PDE solver that treats a low-budget Monte Carlo solution as a structured estimator of the true field and learns a single-pass neural correction to recover a high-fidelity solution. MC$^2$ matches the accuracy of solutions using over $1000\times$ more Monte Carlo samples, outperforming all evaluated mesh-free classical, denoising, and neural-operator baselines. To enable reproducible study of finite-compute PDE solving, we additionally release PDEZoo, the largest standardized elliptic PDE benchmark to date: 2 million PDEs spanning 5 elliptic families and unlimited geometric compositions, with analytic ground truth and multi-budget Monte Carlo trajectories. Together, MC$^2$ and PDEZoo (1) empirically establish that finite-sample Monte Carlo error is structured, learnable, and correctable in a single forward pass, (2) show that we can solve PDEs $\sim$218$\times$ faster than WoS at matched accuracy and achieve a $\sim$171$\times$ reduction in inference time relative to state-of-the-art diffusion methods while attaining higher accuracy, and (3) provide the reproducible evaluation infrastructure the field has so far lacked. Code & Data: https://mc-error-correction.github.io

cs.LG · cs.AI · cs.CE · cs.CV · math.NA

新论文 · v1arXiv:2611.13332

FP64 Performance and the Limits of Ozaki Emulation on HPC Systems

Katarzyna, Swirydowicz, Nicholas Malaya, Noah Wolfe, Paul Bauman, Pieter Ghysels, Jonathan Belof

原文摘要

A growing industry narrative holds that native double precision can be deprioritized in HPC accelerators, on the hypothesis that low-precision tensor cores can emulate FP64 on demand, and that most HPC workloads are bandwidth-bound. This paper evaluates some of those claims and explores the importance of FP64 in scientific computing. We show roofline profiling data of representative HPC workloads, across a variety of common computational motifs, to evaluate the arithmetic-intensity and accuracy requirements of these workloads. We also consider FP64 emulation via the Ozaki method to ascertain some of the possibilities and limitations of these approaches. These results indicate that Ozaki emulation should not be evaluated as a general replacement for FP64, but as an optimization for the subset of workloads whose dominant computation is sufficiently large, dense, and GEMM-shaped. Our analysis indicates this is less than 10\% of the HPC ecosystem. Furthermore, even in this favorable regime, standalone GEMM speedup is not enough to accelerate (or avoid performance regression) for the application: numerical behavior, workspace requirements, matrix shape, and integration overhead determine whether the benefit survives at application scale. This paper characterizes those boundaries empirically. Furthermore, we show that dense GEMM accounts for only a small fraction of FP64 computation across HPC workloads, leaving most application-level FP64 work beyond the reach of GEMM-based emulation.

cs.DC · math.NA

版本更新 · v2arXiv:2611.00301

The Rise of Low Precision Floating Point

Guy Lemieux, Michael L. Overton

原文摘要

Floating point computing has dominated scientific computations since the 1950s because of its versatile ability to approximate real numbers with both sufficiently high precision and sufficiently great range. Since 1985, two specific formats have become standard across the industry, one with 32 bits which is adequate for many purposes and one with 64 bits which allows numbers with both much higher precision and much greater range for applications where these are needed. However, during the past decade, deep learning has come to dominate computation, and narrower floating point formats with lower precision and reduced range have steadily become more common. In this article we discuss this phenomenon, why it is happening, how various 16-bit, 8-bit and even 4-bit floating point formats have been developed, and efforts to standardize these formats.

cs.MS · cs.AR · cs.LG · math.NA

版本更新 · v2arXiv:2608.10342

Shape optimisation of nonlinear Naghdi shells on discrete geometries

Ado Farsi, Alberto Paganini

原文摘要

We present an automated framework for the shape optimisation of a geometrically nonlinear, five-parameter Naghdi model on faceted triangulations. By recovering directors numerically, we remove the requirement for an analytic or spline parameterisation of the mid-surface, whereas partial selective reduced integration alleviates locking. The residual and tangent of symbolic energies are generated in an automated fashion building on Firedrake's and UFL's symbolic differentiation capabilities. Additionally, UFL's and pyadjoint's algorithmic differentiation enables automated first and second shape derivative evaluations that are verifiably correct via Taylor tests. The forward model agrees with the Sze/Abaqus semi-cylinder reference. A three-model mesh-refinement study indicates that geometry recovery and surface faceting have small effects on the predicted deflection. Numerical experiments show that 100 trust-region iterations suffice to reduce the squared-displacement functional of a semi-cylinder by 98.4% and the elastic strain energy of a bracket by 94.6%. The latter result is validated against a COMSOL benchmark. Non-uniqueness of optimal shapes is demonstrated by optimising initial designs subject to small perturbations. Comparisons with linearised designs distinguish optimisation performance from the accuracy of the predicted nonlinear response.

math.NA · cs.CE

新论文 · v1arXiv:2611.13231

Block-Sparse Gauss-Newton Optimization of Finite Basis Physics-Informed Neural Networks: Sparse Automatic Differentiation Approach

Denis Korolev, Adrian Hill, Jeeeun Lee

原文摘要

In this work, we develop an efficient Gauss-Newton optimization approach for finite basis physics-informed neural networks (FBPINNs). The key ingredient is the overlapping domain decomposition architecture of FBPINNs, which localizes the influence of each subnetwork and induces block sparsity in both the residual Jacobian and the corresponding Gauss-Newton matrix. We characterize this structure and exploit it through sparse automatic differentiation, using graph coloring to partition structurally orthogonal Jacobian columns or rows. This reduces differentiation costs and memory requirements while preserving all structurally nonzero derivative information. The resulting compressed Jacobian representations allow us to assemble the Gauss-Newton system directly in block-sparse form, enabling the use of sparse direct solvers and preconditioned iterative methods. In this way, the otherwise expensive second-order optimization problem is reduced to scalable sparse linear algebra. We demonstrate the approach on multiscale PDEs, where accurate approximation requires rich neural network parametrizations and large numbers of collocation points. Numerical experiments show that our block-sparse Gauss-Newton method attains high accuracy and substantially accelerates optimization relative to first-order baselines while remaining computationally tractable for large problem sizes.

math.NA · cs.LG

新论文 · v1arXiv:2611.12424

Physics-informed neural networks in the small-data regime: A controlled comparison with classical PDE surrogates

Kirill Katsuba, Oleg Rudenko, Egor Bugaev, Alexander Tarakanov

原文摘要

Neural surrogates for engineering models are often trained on large collections of simulated solutions. We investigate whether physics-informed neural networks (PINNs) can achieve accurate predictions from scarce labelled observations by incorporating governing equations and prescribed initial and boundary conditions. We introduce hierarchical loss weighting (H-PINN) and its gradient-direction-aware extension (HO-PINN), which adjust the emphasis on physical constraints according to higher-priority losses. Hierarchical PINNs are compared with Vanilla PINN, tuned gradient boosting, support-vector regression, and Gaussian-process regression on a heat equation and three heterogeneous two-phase-flow benchmarks. Experiments use 20--1000 labelled states and ten paired seeds. Each label is one coordinate--state observation, not a complete simulation. At least one PINN variant achieves the highest mean primary-target \(R^2\) in each interpolation and equation-assisted forecasting comparison at 20 and 50 labels. The advantage for water saturation persists across all tested budgets. Hierarchical weighting reduces variability across runs in selected settings. These results demonstrate that known physics can support accurate neural surrogate predictions from very few solution observations, at substantially higher fitting cost than the tested classical regressors.

cs.CE · cs.AI · math.NA

新论文 · v1arXiv:2611.12361

Law-Invariant Operators for Stochastic Mechanics

Vissarion Papadopoulos

原文摘要

A central objective in stochastic mechanics is to separate deterministic physical behavior from the probability laws governing uncertain inputs. While classical spectral approaches achieve this separation in linear settings, nonlinear responses introduce interactions beyond conventional second-order variability response functions (VRF). Inspired by the spirit of VRFs, this work establishes that, for a fixed physical system, uncertainty propagation is described by a law-invariant operator mapping input realizations to responses, governed by the underlying physics and the deterministic parameters of the system. Different probability laws affect the input sampling and resulting response statistics, but not the mapping operator. We establish this law invariance under complete, invertible coordinate transformations and develop complementary, Fourier, fixed-reference polynomial-chaos (PCE) and finite element method (FEM) transfer formulations. Fourier captures nonlinear cross-frequency interactions and recovers the classical VRF in the appropriate linear limit, PCE describes coupling among stochastic modes, while FEM expresses the deterministic inverse-stiffness action. Neural approximations of these deterministic operators are investigated through analytical and finite-element benchmarks, examining cross-law generalization, the influence of training distributions and stochastic excitation, and the prediction of response fields and statistics. The results demonstrate the potential for reusing learned physical operators across different stochastic laws without retraining, while highlighting the importance of adequate training coverage and the challenges of accurately estimating low-probability events.

stat.CO · cs.CE · math.NA

新论文 · v1arXiv:2611.12162

Same Physics, Different Gradients: A State-of-the-Art Report on Differentiable Physics Simulation

Lei Shu, Zizhou Huang, Yunuo Chen, Yifei Li, Tao Du, Minchen Li, Tuur Stuyck, Miles Macklin 等 18 位作者

原文摘要

Differentiable simulation propagates derivatives through a physics simulator, turning animation, system identification, control, and computational design into gradient-based optimization. Engines differ in the physics they model and, less visibly, in which mathematical object they differentiate. We separate two questions the literature conflates. The referent is which object the returned derivative belongs to: the finite computation that executed, the converged equation it approximates, or the continuous dynamics. The realization describes the organization of the backward pass: automatically generated differentiation, an equation-level adjoint solve, or a custom adjoint. The referent defines the target against which exactness is assessed; the implementation determines how accurately that target is approximated. Cost depends on the realization together with how the states needed by the backward pass are retained or recovered. Crossing the axes places every method and engine we classify in a cell, separates claims the literature merges, and shows that the apparent trade-off between exactness and memory follows from how methods have been built rather than from a constraint. A second, orthogonal taxonomy covers contact, where the map is non-smooth. We then ask whether a correct gradient helps. Four mechanisms by which useful gradients are lost are cataloged and measured on one testbed, where the advantage of the exact derivative over the equation-level adjoint depends on where the fitting target comes from as much as on the solver budget, and we review the evidence that zeroth-order methods can outperform simulator gradients in the regimes where those mechanisms act. The model classes, systems techniques and applications are surveyed through this lens, and we close with a reporting checklist, a released diagnostic that computes it, and a research agenda.

cs.GR · cs.CV · cs.LG · cs.RO · math.NA

版本更新 · v2arXiv:2605.06281

INEUS: Iterative Neural Solver for High-Dimensional PIDEs

Jean-Loup Dupret, Davide Gallon, Patrick Cheridito

原文摘要

In this paper, we introduce INEUS, a meshfree iterative neural solver for partial integro-differential equations (PIDEs). The method replaces the explicit evaluation of nonlocal jump integrals with single-jump sampling and reformulates PIDE solving as a sequence of recursive regression problems. Like Physics-Informed Neural Networks (PINNs), INEUS learns global solutions over the entire space-time domain, yet it offers a more efficient treatment of nonlocal terms and avoids the computationally expensive differentiation of full PIDE residuals. These features make INEUS particularly well suited for high-dimensional PDEs and PIDEs. Supported by a contraction-based convergence proof for linear PIDEs, our numerical experiments show that INEUS delivers accurate and scalable solutions for various high-dimensional linear and nonlinear examples.

cs.LG · math.NA · q-fin.CP

新论文 · v1arXiv:2611.12054

Low-rank Low Precision Training of Gaussian Processes

Moritz Poguntke, Eda Oktay, Yuanzhe Xi, Erin Carson, Martin Stoll

原文摘要

Training Gaussian processes (GPs) at large scale is computationally challenging because evaluating the log-marginal likelihood and its gradients involves large, dense kernel matrices. We develop a scalable low-rank GP training approach that combines AnchorNet landmark selection with a fully independent training conditional (FITC) correction. The resulting objective and its analytic gradients can be evaluated using low-rank factors, diagonal quantities, and small dense matrices, without forming the full kernel matrix or its derivatives. The complete training procedure is implemented in single precision and evaluated on nine large-scale regression datasets. Across these problems, the method reaches the reference predictive accuracy while substantially reducing training time, although the required approximation rank varies across datasets. We also study the numerical behavior of the full-kernel predictive solve and show that single precision can become limiting for ill-conditioned problems. Finally, we investigate further reducing selected matrix operations to fp16 or TF32. These lower-precision variants provide only modest additional speedups and can become unreliable when the reduced system is poorly conditioned.

math.NA · cs.LG

新论文 · v1arXiv:2611.11905

On the Global Convergence of Stochastic Adam for Physics-Informed Neural Networks

Yihang Gao

原文摘要

The Adam optimizer is widely used to train neural networks with highly nonlinear and nonconvex objective functions. Remarkably, Adam often drives the empirical training loss nearly to zero. This phenomenon has been partially explained by neural tangent kernel (NTK) analyses of gradient descent (GD) and stochastic gradient descent (SGD). However, analyzing Adam presents additional challenges. Its first-moment momentum couples stochastic gradients across iterations, while its adaptive second-moment preconditioner depends on the same stochastic gradient used in the update. In this paper, we establish linear convergence guarantees for Adam in training sufficiently wide two-layer neural networks with either full-batch or minibatch gradients. For full-batch Adam, we construct a Lyapunov function to control the effect of historical momentum. For minibatch Adam, our analysis builds on an exact virtual-iterate representation, a conditional analysis of the dependence between stochastic gradients and adaptive preconditioners, and uniform concentration bounds for the second-moment estimates. We prove geometric decay of the empirical loss for full-batch Adam and geometric decay in restricted expectation on a high-probability event for minibatch Adam. Beyond regression, we extend our convergence analysis to physics-informed neural networks (PINNs) for solving the Poisson equation, where differential operators and separate sampling of interior and boundary points introduce additional challenges. We establish analogous global convergence guarantees for both deterministic and stochastic Adam under explicit conditions on network width and Adam's parameters, characterizing the effects of minibatch size on convergence. These results help bridge the gap between Adam's empirical success and its theoretical understanding, providing sharper convergence guarantees for PINNs than existing analyses.

cs.LG · math.NA · math.OC

版本更新 · v2arXiv:2606.29129

Improved Scaling for Fast Mode of Ozaki Scheme II

Shota Kawakami, Daisuke Takahashi

原文摘要

Ozaki scheme II emulates high-precision matrix multiplication using low-precision integer matrix operations based on the Chinese remainder theorem (CRT). It first scales the high-precision matrices to convert them into integer matrices. For this scaling step, Ozaki scheme II provides two modes: accurate mode, which uses INT8 matrix multiplication to estimate scaling factors, and fast mode, which applies the Cauchy--Schwarz inequality at lower computational cost. We show that the existing formula lacks scale invariance; multiplying the input matrices by a constant changes the effective bit width of the integer matrices in the scaling step, causing accuracy degradation or CRT recovery failure. To address this, we propose a revised scaling formula derived from the CRT uniqueness condition via the Cauchy--Schwarz inequality. The proposed formula is scale-invariant by construction, guarantees that the CRT uniqueness condition is always satisfied, and introduces no additional overhead over the original fast mode. Experiments on an NVIDIA GH200 GPU show that the proposed method achieves accuracy comparable to that of accurate mode while maintaining throughput comparable to that of fast mode. In the accuracy--throughput trade-off, the proposed method overcomes the accuracy limitation of fast mode and the throughput constraint of accurate mode, offering a superior accuracy and performance.

cs.MS · cs.DC · math.NA

新论文 · v1arXiv:2611.11805

Operator Learning of Microstructure-Dependent Internal-Variable Models for Inelastic Homogenization: Approximation and Complexity

Lianghao Cao, George Stepaniants, Andrew M. Stuart

原文摘要

We study the learning of homogenized constitutive laws for inelastic heterogeneous materials, aiming to replace costly concurrent unit-cell simulations with surrogates that generalize across loading histories and microstructures. We focus on the recurrent neural operator framework, motivated by the explicit form of homogenized Kelvin--Voigt linear viscoelasticity. In this framework, internal variables enable a Markovian approximation of loading-history dependence, and neural operators represent the microstructure-dependent constitutive response. For periodic Kelvin--Voigt viscoelasticity, we prove a universal approximation theorem showing that the model can approximate the homogenized constitutive law to any prescribed accuracy, uniformly in time and over a wide set of admissible strain trajectories and microstructural property fields, in two and three dimensions. This comprises a significant extension of the existing, somewhat limited, one-dimensional theory that uses explicit solutions for piecewise constant materials. Instead, in two and three dimensions, we adopt a more versatile approximation based on the spectral representation of the memory kernel that arises from homogenization. This also enables us to give an explicit bound on the internal-variable dimension, sufficient for approximation, exhibiting logarithmic growth in the inverse relative stress error and in material contrast. Numerical experiments on viscoelasticity and crystal plasticity complement the theory, illustrating the roles of internal-variable dimension and sample complexity in learning homogenized responses from unit-cell data. Theoretical results are formally verified in Lean 4.

math.NA · cond-mat.mtrl-sci · cs.LG · math.DS

版本更新 · v2arXiv:2605.15399

Breakeven complexity: A new perspective on neural partial differential equation solvers

Yijing Zhang, Nicholas Roberts, Tanya Marwah, Mikhail Khodak

原文摘要

Neural surrogate solvers of partial differential equations (PDEs) promise dramatic speedups over numerical methods, especially in scenarios requiring many solves. However, current accuracy-based evaluations do not fully consider two central issues: (1) neural solvers incur substantial up-front costs for data generation, training, and tuning; and (2) classical solvers can also generate low-fidelity solutions at a sufficiently low simulation cost. To explicitly account for these realities and fully incorporate end-to-end costs, we propose an evaluation framework centered on breakeven complexity, a metric that counts the forward solves before a learned solver is cost-effective relative to an error-equivalent traditional solver. To evaluate this measure, we apply scaling laws to determine how much training budget to allocate to data generation and discuss how to achieve smooth error-matching in diverse settings. We evaluate the breakeven complexity of multiple neural PDE solvers on three PDEs on 2D periodic domains from APEBench and a novel benchmark of flows past multiple obstacles generated by the GPU-native PyFR code. Among other findings, our results suggest that neural PDE solvers become more effective as problems get harder in terms of cost, dimension, rollout, physics regime (e.g. higher Reynolds number), etc.

cs.LG · cs.AI · math.NA · physics.comp-ph

新论文 · v1arXiv:2611.11624

Karhunen-Loève Transforms of Autoregressive Windows Based on Matched Symmetry Groups

Mitchell A. Thornton

原文摘要

The Karhunen-Loève transform of a window of a stationary autoregressive process of any order $p$ is described through its poles, extending the Algebraic Diversity framework, which selects a transform by the matched group. For simple roots every basis vector is a sum of at most $p$ sinusoids, at the frequencies where the spectrum equals the eigenvalue, plus modes decaying from the ends. The window covariance differs from a member of the class diagonalized by the type-II discrete cosine transform by a closed-form matrix of rank $2p$, exactly when no pole is zero, so the eigenvalues interlace spectrum samples on the cosine grid, and the coding-gain loss of the cosine transform is an exact function of that matrix with computable bounds: for the $8\times8$ Markov block of image coding at pole $0.95$ it is below 1.2 percent. By Kronecker's theorem the rank is at most $2\max(p,q)$ for rational spectra; a moving-average window departs in its Hankel corners, and an integrated (ARIMA) window departs by low rank from the type-VII sine class, exact for the random walk. The exact symmetry is generically the reflection and contains a wreath product for interleaved windows; the dihedral group of the even extension is matched approximately, the dominant pole's sign choosing cosine or sine. The two groups give the transform a fast form: a parity fold, half-length cosine cores and a cascade of at most $p$ first-order Cauchy stages, exactly $p$ for a minimal AR($p$) model. The first-order case recovers results obtained by other means.

eess.SP · cs.IT · math.NA

新论文 · v1arXiv:2611.11277

Variance-Reduced Stochastic Hessian-Driven Nesterov Acceleration with Adam-Style Preconditioning

Yaxin Yu, Long Chen, Minfu Feng

原文摘要

Stochastic first-order methods use sampled gradients to reduce per-iteration costs, but estimation errors can slow convergence and, in some settings, cause divergence. To improve gradient accuracy while retaining acceleration, we develop scalar and diagonal versions of VR-Adam-SHANG for smooth stochastic convex optimization. The methods combine STORM-type recursive estimation, Adam-style preconditioning, and SHANG's accelerated interpolation and correction, using at most two stochastic gradient evaluations per iteration without full-gradient refreshes. Under unbiased sampling, bounded variance, and mean-square smoothness of gradient noise, polynomial steps with prescribed initialization yield $O(1/K)$ mean-square gradient-estimation error and expected objective-gap bounds combining $O(1/K^2)$ initial-energy decay with an $O(1/\sqrt K)$ stochastic term after $K$ iterations. These guarantees require neither monotonicity of the preconditioner nor a uniform positive lower bound on its eigenvalues. The analysis also establishes polynomial-step convergence of projected nonrecursive Adam-SHANG-s, filling a gap in its original theory. Experiments compare the methods at matched sample-gradient budgets. In every tested setting, at least one of the two versions achieves the lowest mean terminal objective gap among all compared methods.

math.OC · cs.LG · math.NA

版本更新 · v6arXiv:2603.10277

Estimating Condition Number with Graph Neural Networks

Erin Carson, Xinye Chen

原文摘要

In this paper, we propose a fast method for estimating the condition number of sparse matrices using graph neural networks (GNNs). For efficient deployment of GNNs, we introduce a graph feature construction with $\mathrm{O}(\mathrm{nnz} + n)$ complexity, where $\mathrm{nnz}$ is the number of non-zero elements in the matrix and $n$ denotes the matrix dimension. We propose two schemes for estimating the matrix condition number using GNNs; one follows by decomposing the condition number and predicts the relatively more computationally intensive part $\|\mathbf{A}^{-1}\|$, without explicitly forming the inverse, while the other is to predict the whole condition number $κ$. Our approach can be extended to an arbitrary norm. Extensive experiments are conducted for the estimation of the 1-norm and 2-norm condition numbers, which show that our method achieves a significant speedup over the traditional numerical estimation methods. Our software for GNN condition number estimator is made publicly available at https://github.com/inEXASCALE/sparse-kappa.

cs.LG · math.NA

版本更新 · v2arXiv:2605.05435

Active Learning for Conditional Generative Compressed Sensing

Alexander DeLise, Nick Dexter

原文摘要

Generative compressed sensing uses the range of a pretrained generator as a nonlinear model for recovering structured signals from limited measurements. We study a conditional version of this problem for image recovery from subsampled Fourier measurements using prompt-conditioned generative models. Our framework separates two roles of conditioning: the prompt used to design the sampling distribution and the prompt used to define the recovery model. For ReLU and Lipschitz conditional generators, we prove stable recovery bounds showing that prompt-matched Christoffel sampling retains the same Christoffel complexity constant as existing near-optimal generative compressed sensing theory, while prompt mismatch incurs an explicit compatibility penalty. Experiments with Stable Diffusion show that prompts meaningfully reshape Christoffel sampling distributions and influence image recovery. Overall, our results suggest treating prompts as design variables for sensing, approximation, and recovery, opening new directions for the practical use of conditional generative models in compressed sensing, where conditioning, sampling, and optimization must be considered jointly.

cs.LG · math.NA

版本更新 · v2arXiv:2610.36558

Holomorphic Cauchy Networks for Efficient Stationary Tail Computation in High-Dimensional Reflected Brownian Motions

Kehan Wang

原文摘要

Stationary tail computation for high-dimensional reflected Brownian motions (RBMs) remains computationally challenging. Building on NN4MGF's shared-coordinate basic-adjoint-relationship (BAR) framework, we develop Holomorphic Cauchy Networks (HCNs). Native complex-valued rational Cauchy subnetworks enforce holomorphicity on pole-free domains, removing sampled Cauchy-Riemann penalties and their input-derivative graphs. Exact origin anchoring and boundary-coordinate exclusion enforce structural identities, while anchored interactions represent dependent transforms. Under holomorphic-log assumptions, we establish simultaneous value and first-derivative approximation for the constrained architectures and a mixed-derivative lower bound for coordinate-only representations under nonzero cross-covariance. Closed-form real-axis derivatives and sequential backpropagation further control memory. On the same NVIDIA H100 NVL, the tested 30D HCN configuration uses approximately tenfold less peak allocated memory and has 9.7-fold lower mean update time than NN4MGF; extrapolation to the respective scheduled update counts gives a projected full-schedule update-time ratio of 58.2. With 64 million sampled base inputs per high-dimensional run, HCN reaches 50 dimensions on a laptop GPU with 6.367% mean relative error over thresholds 1 through 8. Degree-matched comparisons yield lower reported mean relative errors over the primary evaluation ranges, including 1.422% versus 6.684% on the 2D Harrison non-product benchmark. On a 10D correlated RBM benchmark, HCN achieves 3.829% mean relative error over thresholds 1 through 8 against a simulation reference, compared with 19.386% for an independent-marginal baseline.

cs.LG · math.NA · math.PR

版本更新 · v3arXiv:2602.00474

Persistent-Transient Policy Evaluation for Markov Chains via Minimal Peripheral Quotients

Yang Xu, Vaneet Aggarwal

原文摘要

We study fixed-policy evaluation for finite Markov chains that may be reducible and periodic. Classical evaluation methods with gain and bias decomposition are not always diagnostic: the gain records only invariant Cesàro averages, while persistent phase-dependent behavior is absorbed into the bias together with genuinely transient effects. We identify the real peripheral invariant subspace $\mathcal{K}(P)$ of the transition matrix $P$ as the source of this ambiguity. Quotienting by $\mathcal{K}(P)$ is the minimal exact quotient that removes all non-decaying modes and makes the remaining dynamics strictly stable. After choosing a gauge projection $Π$ with kernel $\mathcal{K}(P)$, the reward admits a unique decomposition $r = g_Π^\star + (I-P)v_Π^\star$, where $g_Π^\star$ is a persistent regime profile and $v_Π^\star$ is a gauge-fixed transient component. An exact comparison with classical normalized gain and bias shows that the new pair reallocates the same information into a persistent profile and a gauge-fixed representative of the transient quotient. This decomposition reconstructs finite-horizon returns, recovers statewise average reward, admits a transient-cost interpretation, and yields a stable estimator under a generative model.

stat.ML · cs.LG · math.NA

新论文 · v1arXiv:2611.10961

Improving Analytic Constants via Interpretable AI-Guided Lean Proof Refinement

Rahul Saha, Alan Li, Anton Xue, Swarat Chaudhuri, Adam Klivans, Pravesh K. Kothari, Raghu Meka

原文摘要

AI agents are increasingly capable of discovering new mathematical results, yet their reasoning and generated proofs can be difficult to understand and verify. We introduce an interpretable framework for AI-guided mathematical discovery, focused on improving quantitative bounds on analytic constants. Our key insight is that existing proofs often contain slack in intermediate lemmas, which can be tightened without reconstructing the entire argument. To exploit this structure, we represent proofs as partially-formalized Lean blueprints: human-interpretable dependency graphs of definitions, lemmas, and theorems. An AI agent then iteratively refines the Lean code while preserving the blueprint's dependency structure, allowing local improvements to automatically propagate to a verified final bound. This interpretable structure allows humans to inspect the agent's refinements and understand how local improvements propagate to the final theorem. For Grothendieck's constant ($\mathcal{K}_G$), we obtain $$ \mathcal{K}_G \leq \fracπ{2\log(1+\sqrt{2})} - 10^{-17}, $$ an explicit numerical improvement over Krivine's 1977 bound. For Korenblum's constant ($\mathcal{K}_K$), we obtain $$ \mathcal{K}_K \geq 0.356565, $$ improving Wang's 2025 bound of $0.3554$. Our experiments demonstrate the potential of both partial formalization and iterative agentic refinement for interpretable mathematical discovery.

cs.AI · cs.LG · math.NA

新论文 · v1arXiv:2611.10700

Spectral Bias Reduction in Local Gated ReLU Architectures

Victorita Dolean

原文摘要

Neural networks trained by gradient-based methods learn the low-frequency content of a target function faster than its high-frequency content. This spectral bias is well documented, but it is much less clear which architectural choices mitigate it and which cannot. We study the question for shallow ReLU models linear in their trainable parameters, where each mode is learned at a rate given by a Gram-matrix eigenvalue, and for a general symmetric bilinear form $a(\cdot,\cdot)$, so that least-squares regression ($a=$ the $L^2$ inner product) and variational or physics-informed losses ($a=$ the $H^1$ energy form) are two instances of one analysis. Within this framework we propose gated local ReLU dictionaries, obtained by multiplying each ReLU feature by a hat partition-of-unity weight, in the spirit of partition-of-unity finite elements and overlapping Schwarz methods. The resulting Gram matrices are block-banded for any local bilinear form, with a block-Toeplitz interior whose matrix-valued symbol governs the spectrum. The two forms then behave very differently. For $L^2$ regression, the symbol is bounded away from zero, the ratio is uniform in the number of windows, and the measured spread of frequency-dependent learning rates falls from $1.9\times10^{5}$ to $4.5$ at matched model size. For the $H^1$ stiffness matrix, the symbol vanishes at zero frequency: gating leaves the $Θ(N^2)$ conditioning intact and inverts the frequency ordering instead of removing the bias. A preconditioner built from the same interior stencil will fix the spectral bias, showing that architecture alone is not enough. Experiments on $1$D regression and Poisson problems confirm the dichotomy, and the regression effect persists when the hidden-layer parameters are trained as well.

cs.LG · math.NA

版本更新 · v2arXiv:2602.07603

Closed-Form Implicit Neural Representations

Woojin Cho, Junghwan Park

原文摘要

Training implicit neural representations (INRs) to capture fine-scale details typically relies on iterative backpropagation and is often hindered by spectral bias when the target exhibits highly non-uniform frequency content. We propose ELM-INR, a backpropagation-free INR that decomposes the domain into overlapping subdomains and fits each local problem using an Extreme Learning Machine (ELM) in closed form, replacing iterative optimization with stable linear least-squares solutions. This design yields fast and numerically robust reconstruction by combining local predictors through a partition of unity. To understand where approximation becomes difficult under fixed local capacity, we analyze the method from a spectral Barron norm perspective, which reveals that global reconstruction error is dominated by regions with high spectral complexity. Building on this insight, we introduce BEAM, an adaptive mesh refinement strategy that balances spectral complexity across subdomains to improve reconstruction quality in capacity-constrained regimes.

cs.LG · math.NA

版本更新 · v3arXiv:2503.03283

Exploring specialization and sensitivity of convolutional neural networks under simultaneous image augmentations

Pavel Kharyuk, Sergey Matveev, Ivan Oseledets

原文摘要

Drawing parallels with the way biological networks are studied, the authors adapt the treatment--control paradigm to explainable artificial intelligence research and enrich it through multi-parametric input alterations. A framework for investigating internal inference impacted by input data augmentations is proposed to address the real-world scenario with multiple simultaneously applied treatments instead of observing a single effect. The internal changes in network operation are reflected in activation changes measured by variance, which can be decomposed into components related to each augmentation by employing Sobol indices and Shapley values. To facilitate distinguishing model-related specifics from the self-interactions of augmentations, an auxiliary variable responsible for their permutations has been incorporated into the analysis. The resulting quantities enable one to visualize sensitivity to different variables and use them as features for further analysis and processing, including guided masking of activations. In addition, a method for single-class sensitivity analysis is introduced through a pipeline that involves class selection according to their match with the prediction bias generated by targeted damaging of the activations. Three convolutional networks, namely AlexNet, VGG11, and ResNet18, have been evaluated via the designed framework, with two vision transformers, ViT-b-16 and Swin-t, partially complementing the findings. Two sets of augmentations, consisting of five and seven distinct input transformations applied simultaneously, have been utilized as sources of variance in the image domain. Relying on the observed parallels, the authors assume that the developed framework can potentially be transferred to the study of biological neural networks in complex environments, as well as enhance the design of autonomous decision-making systems.

stat.ML · cs.AI · cs.LG · math.NA

新论文 · v1arXiv:2611.10487

GEAR: A GPU-Accelerated Global Solver for Bounded Continuous Nonlinear Programs via Linear Bound Propagation

Duo Zhou, Hesun Chen, Xiangru Zhong, Grani A. Hanasusanto, Huan Zhang

原文摘要

We present a novel GPU-accelerated global solver for constrained nonlinear programming (NLP) inspired by the efficient linear bound propagation framework in neural network (NN) verification. Existing work on NN verification, a problem that can be cast as feasibility detection for a nonlinear objective, demonstrated orders-of-magnitude speedups with the linear bound propagation-based framework when compared to traditional solvers. Extending this paradigm to global nonlinear programming requires addressing many challenges: a global constrained NLP solver must effectively exploit all, possibly unstructured, nonlinear constraints to improve the objective, whereas NN verification typically handles simple input box constraints over a well-structured NN and does not directly optimize an objective function. We address these challenges by fully exploiting the linear bounds produced by bound propagation: these linear bounds can be viewed as relaxations of nonlinear constraints and used to provide lower bounds, check infeasibility, and guide the search for a primal solution. In addition, we propose tighter linear relaxations for bilinear, quotient, and other commonly used nonlinear functions in NLP problems, and a projected augmented Lagrangian method tightly coupled to our dual solving procedure to produce primal solutions. Our method solves 421 of 505 bounded and continuous NLP problems in the GAMS and MINLPLib benchmarks under a 180-second limit, more than each of the six global solvers we compare with, including Gurobi and BARON (commercial), and SCIP (open-source). Sixteen of these problems are certified by none of the six global solvers; all of them contain exponential, logarithmic, or trigonometric terms, which are often challenging for traditional approaches. Project webpage is at https://verified-intelligence.github.io/GEAR.

math.OC · cs.LG · cs.MS · math.NA

新论文 · v1arXiv:2611.09887

关于 ReLU 网络集合的(非)闭性

On the (non-)closedness of the set of ReLU networks

Eloi Martinet, Leon Bungert

AI 速读 基于摘要

作者证明单隐层 ReLU 网络集合在 Sobolev 空间中是弱闭的,而宽度有界的双隐层 ReLU 网络集合在 Sobolev 空间与连续函数空间中均不闭。

cs.LG · math.CA · math.FA · math.NA · math.OC

查看原文摘要

We prove that the set $\mathcal{N}_N$ of ReLU networks with one hidden layer and at most $N\in\mathbb N$ neurons is weakly closed in $W^{1,p}(Ω)$, for a bounded connected open set $Ω\subset\mathbb R^d$. Consequently, every coercive, weakly lower semicontinuous energy on this space attains its minimum over $\mathcal{N}_N$. This covers the Deep Ritz method with convex integrands, Sobolev training and Lipschitz learning. Moreover, as the weight decay parameter tends to zero, minimizers of the regularized problem converge, up to subsequences, to minimizers with minimal parameter norm. We also prove that the set of ReLU networks with two hidden layers of bounded widths is not closed in $W^{1,p}(Ω)$ or in $C(\overlineΩ)$. This disproves a conjecture of Petersen, Raslan and Voigtlaender.

新论文 · v1arXiv:2611.09053

Equidistribution of Solution Gradients in Physics-Informed Neural Networks via Coordinate Transformations

Tianchi Yu, Ivan Oseledets

原文摘要

Physics-Informed Neural Networks (PINNs) often struggle to approximate solutions with sharp gradients due to optimization difficulties and spectral bias. Motivated by the principle of gradient equidistribution from the stretching approach in grid generation, which redistributes solution gradients to reduce functional complexity, we propose Coordinate Transformation PINNs (CTPINNs), which incorporate learnable coordinate transformations to map the physical domain into a computational domain where the physical solutions become approximately affine, making them easier for neural networks to learn. We propose two constructions of the transformation: an explicit construction based on Majorants and an implicit construction based on invertable neural networks. Experimentally, the explicit construction can improve accuracy for PDEs with available explicit coordinate transformations, and the implicit construction can be combined with other advanced PINN models as an efficient plug-in module to improve performance across a range of challenging PDE benchmarks.

math.NA · cs.AI · cs.LG

新论文 · v1arXiv:2611.08910

Task-Contribution Dynamics of Adam in Physics-Informed Neural Networks: Slow-Manifold Analysis and Gradient Modulation

Enzhan Zheng, Xiaoliang Song

原文摘要

Physics-informed neural networks (PINNs) often suffer from imbalanced optimization of competing loss components, particularly when approximating solutions with high-frequency or multiscale features. Although existing balancing strategies have proven effective in alleviating this difficulty, the mechanisms governing the evolution of individual task contributions to Adam's parameter updates remain insufficiently understood. We introduce task-wise indicators to characterize this evolution and identify three distinct training regimes: smooth variation, intermittent spikes, and persistent oscillations. Focusing on the early smooth regime, we combine discrete-time analysis with a continuous-time singular perturbation framework under the full-batch setting and the parameter regime $(1-β_2)/(1-β_1)\ll1$. A first-order approximation of the locally attracting slow manifold reveals two factors governing the step-to-step redistribution of task contributions: curvature-related alignment disparities and differences in the coordinatewise distributions of preconditioned task gradients. Both factors involve Adam's shared second-moment estimate. Inspired by AdaTask, we propose Gradient-Modulated Adam (GMAdam), which modulates incoming task gradients using task-specific second-moment estimates. Numerical experiments on high-frequency Poisson and wave equations, the Allen-Cahn equation, and two-dimensional lid-driven cavity flow ($Re=100,400$) demonstrate consistent improvements in predictive accuracy over AdaTask at comparable computational cost.

math.NA · cs.LG