跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1858新增论文
+1473其中 AI 相关
+199已有论文修订
+59新增 AI 导读

arXiv 最近完成扫描:2026-11-18 04:28 · 全站最近入库:2026-11-18 11:33各信源进度 →

统计口径与数据新鲜度

统计 2026-11-17 12:02 — 2026-11-18 12:02(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-18 04:28最近发现新增版本:2026-11-18 11:33库内论文最新更新时间:2026-11-17 20:55

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

25,964 已收录论文21,631 AI 相关7,801 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 12 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-18 04:27(北京时间)
版本更新 · v2arXiv:2602.19512

Variational Trajectory Optimization of Anisotropic Diffusion Schedules

Pengxi Liu, Zeyu Michael Li, Xiang Cheng

原文摘要

We introduce a variational framework for diffusion models with anisotropic noise schedules parameterized by a matrix-valued path $M_t(θ)$ that allocates noise across subspaces. Central to our framework is a trajectory-level objective that jointly trains the score network and \emph{learns} $M_t(θ)$, which encompasses general parameterization classes of matrix-valued noise schedules. We further derive an estimator for $\partial_θ\nabla \log p_t$ that enables efficient optimization of the $M_t(θ)$ schedule. For inference, we develop an efficiently-implementable reverse-ODE solver that is an anisotropic generalization of the second-order Heun discretization algorithm. Across CIFAR-10, AFHQv2, FFHQ, and ImageNet-64, our method consistently improves FID with largest gains at low NFE. Code is available at https://github.com/kilonova801/anisotropic-diffusion.

cs.LG · cs.CV

版本更新 · v3arXiv:2602.02557

The Alignment Curse: Modality Alignment Supercharges Audio Attacks via Text Transfer

Yupeng Chen, Junchi Yu, Aoxi Liu, Baoyuan Wu, Philip Torr, Adel Bibi

原文摘要

Recent advances in end-to-end trained omni-models have substantially improved audio capabilities by strengthening text-audio modality alignment. However, whether such alignment inadvertently facilitates the transfer of safety vulnerabilities across modalities remains underexplored. This question is critical as text-based jailbreak attacks are considerably more mature than audio-based ones; if they transfer systematically, current audio safety evaluations may underestimate risks originating from the text modality. In this paper, we introduce the Alignment Curse, a formally characterized and empirically validated principle showing that stronger modality alignment enables more effective transfer of attacks from text to audio, revealing a fundamental tension between capability and safety. Motivated by this principle, we conduct a comprehensive black-box evaluation of three attack categories on recent omni-models (e.g., Qwen2.5-Omni, Qwen3-Omni): text attacks, text-transferred audio attacks, and audio attacks. We find that text-transferred audio attacks perform comparably to, and often better than, audio-based attacks, exhibiting a clear advantage under audio-only access. This suggests that text-based vulnerabilities play a pivotal role in shaping audio safety risks. Finally, we empirically analyze the relationship between modality alignment and transfer effectiveness across attack methods and models, observing consistent support for the Alignment Curse: tighter modality alignment leads to more effective cross-modality attack transfer.

cs.LG · cs.AI · cs.SD

版本更新 · v2arXiv:2603.09527

Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation

Luxi Lin, Zhihang Lin, Zhanpeng Zeng, Yuhao Chen, Qingyu Zhang, Jixiang Luo, Xuelong Li, Rongrong Ji

原文摘要

Speculative decoding accelerates LLM inference but suffers from performance degradation when target models are fine-tuned for specific domains. A naive solution is to retrain draft models for every target model, which is costly and inefficient. To address this, we introduce a practical parameter- and data-efficient framework named Efficient Draft Adaptation, abbreviated as EDA, for efficiently adapting draft models. EDA introduces three innovations: (1) a decoupled architecture that utilizes shared and private components to model the shared and target-specific output distributions separately, enabling parameter-efficient adaptation by updating only the lightweight private component; (2) a data regeneration strategy that utilizes the fine-tuned target model to regenerate training data, thereby improving the alignment between training and speculative decoding, leading to higher average acceptance length; (3) a sample selection mechanism that prioritizes high-value data for efficient adaptation. Our experiments show that EDA effectively restores speculative performance on fine-tuned models, achieving superior average acceptance lengths with significantly reduced training costs compared to full retraining. Code is available at https://github.com/Lyn-Lucy/Efficient-Draft-Adaptation.

cs.LG · cs.AI

版本更新 · v3arXiv:2509.03244

FoMEMO: Towards Foundation Models for Expensive Multi-objective Optimization

Yiming Yao, Fei Liu, Liang Zhao, Xi Lin, Yilu Liu, Qingfu Zhang

原文摘要

Expensive multi-objective optimization is a prevalent and crucial concern in many real-world scenarios, where sample-efficiency is vital due to the limited evaluations to recover the true Pareto front for decision making. Existing works either involve repeatedly fitting Gaussian process surrogates from scratch for each newly encountered problem, or rely on computationally demanding hypervolume-oriented policy learning for amortized optimization, making it challenging to achieve scalable pre-training and efficient yet robust adaptation for diverse emerging real-world applications. To address these challenges, we propose FoMEMO (Foundation Models for Expensive Multi-objective Optimization), which adopts a decomposition-based training paradigm that converts multi-objective optimization into preference-wise aggregated posterior learning tasks, enabling scalable pre-training on hundreds of millions of diverse synthetic datasets without relying on extensive real-world domain experiments. At test time, given observed trajectories from unseen problems and user preferences, the foundation model performs in-context posterior prediction and optimizes the derived acquisition functions for efficient candidate generation, achieving strong generalization and optimization performance without any subsequent model training or updates.

cs.LG · cs.AI

新论文 · v1arXiv:2611.16301

LoCoJAX: Host-Local Coordination for Small-Model RL on TPUs

Loki Chen, Gregory B. Rehm, Yang Song, Hongye Xie, Ming Lei, Chongyang Bai, Bob Kamma, Mingfu Liang 等 16 位作者

原文摘要

Reinforcement learning (RL) increasingly targets small language models, and executing these workloads efficiently across multiple accelerators requires effective coordination. On TPUs, Pathways-based RL uses a single controller to coordinate computation across hosts. This execution model is effective for large models, but its coordination computational costs are harder to amortize when small models repeatedly invoke short computations. To characterize these computational costs, we sweep dense models from 0.6B to 405B parameters on TPU v7x under a decode-heavy workload and measure how host and network computational overhead vary with model size. At 0.6B, host orchestration occupies 36% of step time and network communication 2%, whereas for large models, estimated network computational costs dominate the non-compute time. Motivated by these findings, we present Local Coordination JAX (LoCoJAX), which implements local rollout and reward processing on each TPU host with synchronous global training. In controlled comparisons against a tuned and optimized Pathways baseline on Qwen3-0.6B across three workloads and 8-64 TPU v7x chips, its colocated mode (LoCoJAX-CO) is 1.3-3.6x faster per step; on GSM8K, it completes a step faster on 8 chips than Pathways does on 64. Finally, a Ray-managed disaggregated mode (LoCoJAX-DA) overlaps rollout and training on separate hosts, and placement experiments establish when this overlap compensates for dividing devices and transferring weights: only when rollout dominates the step and receives most of the hosts. On NuminaMath-CoT at 256 chips, giving rollout 75% of hosts cuts step time by about 15% compared with LoCoJAX-CO.

cs.LG · cs.DC

版本更新 · v2arXiv:2605.05659

Structural Correspondence and Universal Approximation in Diagonal plus Low-Rank Neural Networks

Ying Chen, Aoxi Li, Jihun Kim, Javad Lavaei

原文摘要

Structured low-rank and sparse-plus-low-rank parameterizations, motivated by methods such as LoRA and low-rank factorization, impose strong structural constraints on the linear transformations within neural networks. We study the expressive consequences of these constrained structures from the perspective of universal approximation. We first characterize that purely low-rank neural networks, despite being able to exactly interpolate arbitrary finite scalar datasets, can fail to approximate continuous functions that vary along directions ignored by the low-rank hidden mapping. We show that augmenting the low-rank structure with a one-parameter diagonal component suffices to overcome this geometric bottleneck, yielding a Diagonal-plus-Low-Rank (DLoR) structure. We show that a full-rank linear transformation can be reconstructed using DLoR components either by expanding network width through additive decomposition or by extending network depth through multiplicative decomposition. We term this correspondence between additive width expansion and multiplicative depth extension the Structural Correspondence framework. By combining these constructions with local linearization of the activation function, we prove that DLoR neural networks restore universal approximation for general activation functions. These results characterize how architectural width and depth can compensate for severe rank constraints without requiring unrestricted dense hidden transformations. Diagonal terms are required for multiplicative construction, while additive expansion works with purely low-rank terms. We further determine its exact cost: both the minimal number of rank-$r$ DLoR factors representing $W$ and the minimal network depth for that layer equal $\lceilκ(W)/r\rceil$, where $κ(W)=\min_{γ\ne0}rank(W-γI)$, achieving a lower cost than its dense representation when $W$ is identity-centered.

cs.LG

新论文 · v1arXiv:2611.11364

OOM: Offload-Optimized Full-Parameter MoE Training via Memory-Execution Co-Design

Ruijia Yang, Shiyuan Lin, Chengxi Liao, Zeyi Wen

原文摘要

Offloading model state to host memory and streaming layer weights let a single multi-GPU node train large mixture-of-experts (MoE) models. GPU memory then becomes a performance budget, shared by resident expert state, retained activations, and execution worksets. The best allocation depends on the expert execution path, batch size, and sequence length. We present OOM (Offload-Optimized MoE), a full-parameter MoE training system that co-designs expert execution and memory allocation. For expert-parallel execution, OOM updates resident experts on their owning GPUs and jointly plans state residency and MoE activation retention according to pipeline dependencies. For local execution, OOM bounds GPU worksets through dependency-aware partitioning; in hybrid models, it also retains attention results on the host to avoid costly recomputation and selectively regenerates layer boundaries through cheaper layers to make room. Calibrated cost models guide joint configuration within GPU and host budgets to minimize step time; OOM selects among the configured paths through short executions. On four H100s, OOM trains with 1M-token contexts, whereas FSDP2, ZeRO-Offload, and MegaTrain exhaust GPU memory at 512K. It also trains a 122B-parameter MoE model, on which all three exhaust host memory. On four 24 GB RTX 4090s, it trains 512K-token contexts and 1,024 1K-token sequences per step without gradient accumulation. Across matched-global-batch comparisons, OOM delivers 1.58x and 4.08x geometric-mean (up to 2.14x and 7.77x) throughput over CPU-offloaded FSDP2 on the two platforms while using the least host memory among the evaluated systems -- as little as 1/3.0 of FSDP2's.

cs.DC · cs.LG

版本更新 · v3arXiv:2603.05093

From Baselines to Transport Geodesics: Axiomatic Attribution via Optimal Generative Flows

Cenwei Zhang, Lin Zhu, Manxi Lin, Lei You

原文摘要

Feature attributions often hide a critical modeling choice: they explain a prediction along a counterfactual path from a reference state to an input. Different baselines, interpolations, and generative trajectories define different paths and can therefore produce different explanations. We study this path ambiguity as a modeling problem. Our central question is whether the path can be chosen by a distribution-level transport process, rather than by a hand-designed interpolation or by the sensitivity geometry of the model being explained. We separate attribution into fixed-path credit allocation and path selection. For a fixed path, we prove that the Aumann-Shapley line integral is the unique attribution rule under fixed-path axioms and coordinate-trace separability. For path selection, we minimize kinetic action over flows that transport a reference distribution to the data distribution, yielding a transport-geodesic attribution principle. We instantiate this framework using Rectified Flow and derive stability bounds quantifying how vector-field approximation errors affect the resulting attributions. Experiments show that Reflow improves cross-seed attribution stability, while learned transport paths yield spatially coherent explanations with competitive deletion faithfulness among the evaluated path-based methods. Our code is available at https://github.com/cenweizhang/OTFlowSHAP.

cs.LG · cs.AI · cs.CV

版本更新 · v4arXiv:2608.05466

Recursive Synthesis for Long-Horizon Terminal Tasks

Zhongzhi Li, Yucheng Shi, Zongxia Li, Ruhan Wang, Anhao Li, Jiaxi Li, Junyao Yang, Lei Ke 等 14 位作者

原文摘要

High-quality long-horizon training data for terminal agents is costly and difficult to scale, as manual authoring is expensive and direct LLM generation often produces inconsistent instructions, environments, solutions, and verifiers. We present Recursive Synthetic Terminal Tasks (RST), a recursive verified synthesis framework for constructing long-horizon terminal-agent tasks at scale. Starting from verified seed tasks, RST extends the reference solution, realigns the verifier and instruction to the new workflow, validates the result in a fresh sandbox, and reuses accepted tasks as seeds for subsequent rounds. Across fifteen recursive rounds, RST produces 37,484 synthesized terminal-agent tasks at roughly $0.05 per task. Task difficulty increases substantially over rounds: the median reference solution grows from 67 to 374 lines, the median number of executed commands grows from 40 to 244, and DeepSeek-V4-Pro pass@4 drops from 90% at R1 to 2.5% at R15. To demonstrate training utility, we collect rejection-sampled Qwen3.5 trajectories on the synthesized tasks and use them for supervised fine-tuning. Fine-tuning on these trajectories improves Qwen3.5-27B and Qwen3.5-122B-A10B by up to 10 points on Terminal-Bench 2, Terminal-Bench Hard, and Long-Horizon Terminal Bench, while agentic PPO lifts Qwen3.5-27B to 49.44%, 32.00%, and 22.07% on the three benchmarks, corresponding to relative gains of 20.0%, 41.2%, and 21.9% over the base model. Moreover, after 15 rounds, the recursion shows no ceiling: synthesis yield and validation rates remain stable as difficulty keeps climbing, indicating that the process can continue well beyond the scale reported here.

cs.AI · cs.LG

新论文 · v1arXiv:2610.39409

Safe Multi-Objective Reinforcement Learning: Zero Constraint Violation and Primal-Averaging Convergence

Chenxi Li, Yao Shu, Siyang Gao, Shuang Qiu, Guangxin Jiang

原文摘要

Many applications of multi-objective reinforcement learning (MORL) require policies that balance conflicting objectives while satisfying safety constraints. We study safe MORL with zero constraint violation throughout learning, covering both cumulative and stepwise constraints. A key challenge is to explore sufficiently under unknown dynamics and constraint costs without violating the constraints. For a given preference over objectives, we develop an algorithm to learn a safe Pareto optimal policy by combining optimistic reward estimates with pessimistic cost bounds. With high probability, every policy executed over $T$ episodes incurs zero constraint violation. The algorithm returns a safe Markov policy induced by the averaged occupancy measure and achieves an $\widetilde{O}(1/\sqrt{T})$ suboptimality gap under smooth Tchebycheff scalarization. We further develop a preference-free algorithm that first explores safely without a prespecified preference and then computes a safe policy for any given preference without further interaction. Over $T$ exploration episodes and $K$ planning iterations, it retains the same safety guarantee and achieves an $\widetilde{O}(1/\sqrt{T}+1/\sqrt{K})$ suboptimality gap. Both algorithms require knowledge of a policy that strictly satisfies the constraints and its costs. We then relax these requirements, incurring at most constant cumulative violation. To our knowledge, we provide the first sample-efficient MORL algorithms that guarantee safety for every executed policy.

cs.LG · cs.AI · math.OC · stat.ML

新论文 · v1arXiv:2610.34834

The Noise-Transmission Burden of Diffusion Network: Why Image Prediction Can Outperform Noise Prediction

Gyeonghun Kang, Pengxi Liu, Bingshuo Qian, Xiang Cheng

原文摘要

Clean-image prediction ($x$-prediction) and noise prediction yield equivalent population-optimal denoisers, yet can perform very differently in finite diffusion networks. For noisy input $Y_t$ with noise standard deviation $σ_t$, a noise predictor $f_{\varepsilon,t}$ induces the clean-image estimate $\widehat X_1=Y_t-σ_t f_{\varepsilon,t}(Y_t)$. This reveals a noise-transmission burden: the predictor must preserve information needed to cancel the input corruption. Under Gaussian corruption of $d$-dimensional data, a linear input bottleneck of rank $r<d$ imposes a denoising-error floor $σ_t^2(d-r)$ even with unrestricted downstream computation, while the corresponding optimal risk for image prediction remains bounded. EDM attenuates this bottleneck contribution at high noise; exact preconditioning cancels it at every noise level. Without an input bottleneck, assuming exact affine recovery of the noisy patch from the model representation, we characterize how the covariance spectrum of corrections beyond linear denoising determines the width required above the patch dimension. ImageNet experiments with JiT show that measured denoising losses reflect the predicted bottleneck floor and its attenuation or cancellation through preconditioning. Further widening yields larger relative denoising gains at low noise, where corrections beyond linear denoising spread across more directions, consistent with our spectral characterization of the required model width. Together, these results provide a denoising-based account of practical generation-quality gaps across prediction targets, preconditioning choices, and model widths.

stat.ML · cs.LG

版本更新 · v4arXiv:2402.03941

Discovering and Reasoning of Causality in the Hidden World with Large Language Models

Chenxi Liu, Yongqiang Chen, Tongliang Liu, Mingming Gong, James Cheng, Bo Han, Kun Zhang

原文摘要

Revealing hidden causal variables alongside the underlying causal mechanisms is essential to the development of science. Despite the progress in the past decades, existing practice in causal discovery (CD) heavily relies on high-quality measured variables, which are usually given by human experts. In fact, the lack of well-defined high-level variables behind unstructured data has been a longstanding roadblock to a broader real-world application of CD. This procedure can naturally benefit from an automated process that can suggest potential hidden variables in the system. Interestingly, Large language models (LLMs) are trained on massive observations of the world and have demonstrated great capability in processing unstructured data. To leverage the power of LLMs, we develop a new framework termed Causal representatiOn AssistanT (COAT) that incorporates the rich world knowledge of LLMs to propose useful measured variables for CD with respect to high-value target variables on their paired unstructured data. Instead of directly inferring causality with LLMs, COAT constructs feedback from intermediate CD results to LLMs to refine the proposed variables. Given the target variable and the paired unstructured data, we first develop COAT-MB that leverages the predictivity of the proposed variables to iteratively uncover the Markov Blanket of the target variable. Built upon COAT-MB, COAT-PAG further extends to uncover a more complete causal graph, i.e., Partial Ancestral Graph, by iterating over the target variables and actively seeking new high-level variables. Moreover, the reliable CD capabilities of COAT also extend the debiased causal inference to unstructured data by discovering an adjustment set. We establish theoretical guarantees for the CD results and verify their efficiency and reliability across realistic benchmarks and real-world case studies.

cs.LG · cs.AI · stat.ME