跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.25717 · v1

VR-SparseQuant: Accurate FP4 Quantization with 2:4 Sparsity on Vera Rubin

首次提交 2026-11-19 16:29
版本更新 2026-11-19 16:29
雷达观测 2026-11-20 19:58
arXiv ↗PDF ↗

摘要

In recent years, GPUs have incorporated increasingly specialized support for low-precision computation, from Hopper's FP8 to Blackwell's FP4 Tensor Cores. Precision formats have evolved alongside this hardware, from MXFP4 and NVFP4 on Blackwell to E2M1 FP4 with direct UE5M3 block scaling on Vera Rubin, which we term as VRFP4\boldsymbol{VRFP4}. These advances help serve larger language models at lower cost, but four-bit activation quantization can substantially degrade model quality. We introduce VR-SparseQuant, which uses Vera Rubin's elementwise 2:4 sparsity to recover accuracy while retaining native FP4 execution. By allowing two sparse streams to overlap, we use the same FP4 value budget as dense quantization to refine activations with larger errors. Offline scale-ratio calibration avoids runtime search, while a shared-weight GEMM accumulates both streams efficiently. On Nemotron-3-Nano-30B-A3B, VR-SparseQuant reduces WikiText-2 perplexity from 6.85\boldsymbol{6.85} to 5.57\boldsymbol{5.57} versus dense RTN W4A4. Including activation preparation, our native Rubin kernels for VRFP4 achieve up to 3.1×\boldsymbol{3.1\times} GEMM and 2.7×\boldsymbol{2.7\times} MoE speedup over BF16. To the best of our knowledge, this is the first study of adaptive activation quantization using Vera Rubin's elementwise 2:4 structured sparse FP4 instructions.

作者备注:16 Pages, 7 Figures

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-19 16:29 · 最新观测版本

VR-SparseQuant: Accurate FP4 Quantization with 2:4 Sparsity on Vera Rubin

In recent years, GPUs have incorporated increasingly specialized support for low-precision computation, from Hopper's FP8 to Blackwell's FP4 Tensor Cores. Precision formats have evolved alongside this hardware, from MXFP4 and NVFP4 on Blackwell to E2M1 FP4 with direct UE5M3 block scaling on Vera Rubin, which we term as VRFP4\boldsymbol{VRFP4}. These advances help serve larger language models at lower cost, but four-bit activation quantization can substantially degrade model quality. We introduce VR-SparseQuant, which uses Vera Rubin's elementwise 2:4 sparsity to recover accuracy while retaining native FP4 execution. By allowing two sparse streams to overlap, we use the same FP4 value budget as dense quantization to refine activations with larger errors. Offline scale-ratio calibration avoids runtime search, while a shared-weight GEMM accumulates both streams efficiently. On Nemotron-3-Nano-30B-A3B, VR-SparseQuant reduces WikiText-2 perplexity from 6.85\boldsymbol{6.85} to 5.57\boldsymbol{5.57} versus dense RTN W4A4. Including activation preparation, our native Rubin kernels for VRFP4 achieve up to 3.1×\boldsymbol{3.1\times} GEMM and 2.7×\boldsymbol{2.7\times} MoE speedup over BF16. To the best of our knowledge, this is the first study of adaptive activation quantization using Vera Rubin's elementwise 2:4 structured sparse FP4 instructions.

查看此版本解读 →查看此版本原文 ↗