arXiv:2608.06630 · v2
The Sparsity Whisperer
摘要
Pruning reduces the inference cost of large language models, but existing criteria primarily preserve large activations or reconstruct layer outputs. We argue that this overlooks a key computation performed by particularly sparsity-sensitive neurons in the MLP up and gate projections: separating similar inputs into dissimilar outputs. This suggests that effective pruning should preserve not only activations, but also the differences between outputs more broadly. We introduce a family of difference-informed pruning methods built upon this principle. Wisp+ is a neuronwise, update-free method that prioritizes the input pairs each neuron separates most strongly relative to their distance. Wisp provides a more efficient layerwise alternative that scores weights using shared input-difference norms. Finally, Whisper extends global difference preservation to second-order reconstruction using a lightly regularized difference Hessian. Across Llama 2 and 3.1 models from 7B to 405B parameters, our second-order variant consistently improves over strong reconstruction-based baselines, while our update-free variants improve over activation-aware baselines, especially in constrained settings. The improvements over Wanda and SparseGPT extend to structured sparsity, downstream evaluations, and other model families. Augmenting stronger techniques such as RIA and ALPS with our difference-informed criteria yields further improvements, shifting the overall accuracy-runtime frontier outward at negligible additional cost. These results suggest that preserving output differences is a broadly useful and composable signal for post-training LLM sparsification.
作者备注：Accepted to the Fortieth Annual Conference on Neural Information Processing Systems (NeurIPS 2026). 10 pages, 3 figures. Code available at https://github.com/Shavit-Lab/Whisper