Two-party privacy-preserving Transformer inference today mainly follows one of two paradigms: non-interactive computation over fully homomorphic encryption (FHE), or a hybrid of FHE and secure multi-party computation (MPC). Either way, linear layers account for a large share of the compute and communication budget. Most existing systems attack this cost by designing bespoke plaintext encodings and ciphertext matrix algorithms that minimize the expensive key-switching operations. The price is tight coupling: each scheme only works with its own packing/encoding strategy, and moving data between linear and nonlinear layers still triggers online ciphertext transmission and ring-field conversion. We take a different route. Instead of optimizing within a chosen encoding, we design a series of general algorithms for linear layers that can be readily integrated into existing inference systems, based on FHE and additive secret sharing. Our algorithms work with any compatible RLWE-based scheme and plug into both non-interactive FHE and hybrid FHE-MPC pipelines. Output interfaces in proposed algorithms allow them to be composed with different downstream nonlinear layers. Relative to existing linear layer schemes, our construction removes online key switching, ciphertext transmission, and online ring-field conversion. As a complement, we further propose a fused RNS-CKKS technique that reduces the computational cost of multiplication followed by rotation. We prove correctness and semi-honest security of the proposed algorithms and quantify the resulting savings analytically.
Zhenhua Liu Soochow University, Lijun Li Shanghai Artificial Intelligence Laboratory, Ruizhe Chen, Yuxian Jiang, Tong Zhu Shanghai Artificial Intelligence Laboratory, Zhaochen Su, Wenliang Chen Soochow University, China, Jing Shao Shanghai AI Laboratory
While guided decoding, especially value-guided methods, has emerged as a cost-effective alternative for controlling language model outputs without re-training models, its effectiveness is limited by the accuracy of the value function. We identify that this inaccuracy stems from a core distributional gap: existing methods train static value functions on trajectories sampled exclusively from the base policy, which inherently confines their training to a narrow and suboptimal view of the potential output space. We propose Iterative Value Refinement, a novel framework designed to bridge this gap. It employs Value Exploration to provide a more comprehensive and robust training signal, complemented by Iterative Self-Refinement, which uses the improved value function from one iteration to guide the generation of higher-quality data for the next. Extensive experiments on text summarization, multi-turn dialogue, and instruction following demonstrate the effectiveness of our framework in aligning language models. Our approach not only achieves alignment but also significantly reduces computational costs by leveraging principled value function optimization for efficient and effective control.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表
Haiduo Huang, Fuwei Yang AMD Inc., Zhenhua Liu, Pengju Ren Xi'an Jiaotong University
Speculative decoding has emerged as a promising technique to accelerate large language model inference by employing a smaller draft model to predict multiple tokens, which are then verified in parallel by the larger target model. However, existing approaches face a fundamental limitation: candidates at the same tree layer share identical feature representations, constraining diversity and diminishing overall effectiveness. We identify this as an intra-layer coupling problem that limits prediction accuracy. To address this challenge, we propose Jakiro, which introduces decoupled Mixture of Experts (MoE) into the draft model, enabling different experts to generate diverse candidate tokens from distinct feature spaces. We further propose Contrastive-Enhanced Parallel Decoding (CEPD) that combines autoregressive and parallel decoding with a contrastive mechanism to reduce inference steps while maintaining accuracy. Extensive experiments across diverse models and tasks demonstrate that Jakiro achieves significant speedups over strong baselines, with particularly notable improvements in non-greedy decoding scenarios where token diversity is crucial.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表