← 研踪 PaperTrailarXiv:2605.17997 · v2
MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantization
首次提交 2026-05-18 15:51
版本更新 2026-11-07 11:41
雷达观测 2026-11-10 10:11
摘要
Recently, residual reconstruction-based model quantization methods have achieved promising performance in low-bit Post-Training Quantization (PTQ) by introducing cross-module residuals to reduce error accumulated from previous modules. However, these residuals may also introduce additional bias arising from the Hessian-Approximation (HA) assumption, resulting their two-sided effect and potentially leading to suboptimal quantization performance. Therefore, in this work, we propose a residual regulation strategy to mitigate the HA bias associated with residual strength, while preserving accumulated-error correction. Furthermore, we observe that this two-sided effect is module-dependent, we futher propose a Module-Adaptive Residual Reconstruction (MARR), which assigns a module-specific scaling coefficient to adaptively balance accumulated-error correction and residual-related HA bias for each module. To avoid exhaustive per-module coefficient search and obtain a stable coefficient estimate, we design an Proportional-Integral-Derivative (PID)-based update strategy that employs reconstruction error as feedback to progressively refine this coefficient. Experiments on several typical Large Language Models (LLMs) and Vision Transformers (ViTs) demonstrate the effectiveness of MARR under low-bit quantization ($\leq$4-bit), achieving up to 20.2\% perplexity reduction on LLMs and up to 4.6\% accuracy gains on ViTs over the residual reconstruction baselines. Code will be made publicly available upon acceptance.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-07 11:41 · 最新观测版本
MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantization
Recently, residual reconstruction-based model quantization methods have achieved promising performance in low-bit Post-Training Quantization (PTQ) by introducing cross-module residuals to reduce error accumulated from previous modules. However, these residuals may also introduce additional bias arising from the Hessian-Approximation (HA) assumption, resulting their two-sided effect and potentially leading to suboptimal quantization performance. Therefore, in this work, we propose a residual regulation strategy to mitigate the HA bias associated with residual strength, while preserving accumulated-error correction. Furthermore, we observe that this two-sided effect is module-dependent, we futher propose a Module-Adaptive Residual Reconstruction (MARR), which assigns a module-specific scaling coefficient to adaptively balance accumulated-error correction and residual-related HA bias for each module. To avoid exhaustive per-module coefficient search and obtain a stable coefficient estimate, we design an Proportional-Integral-Derivative (PID)-based update strategy that employs reconstruction error as feedback to progressively refine this coefficient. Experiments on several typical Large Language Models (LLMs) and Vision Transformers (ViTs) demonstrate the effectiveness of MARR under low-bit quantization ($\leq$4-bit), achieving up to 20.2\% perplexity reduction on LLMs and up to 4.6\% accuracy gains on ViTs over the residual reconstruction baselines. Code will be made publicly available upon acceptance.
查看此版本解读 →查看此版本原文 ↗