Amplitude-Only FFN Intervention for Tool-Structured LLM Inference: Method, Gated Evaluation Protocol, and Cross-Model Empirical Results
Sheng Xu, Zhen Chen, Junhua Wang, Boyuan Huang, Ke Jia, Jiadun Zhu, Yiming Xu
We study amplitude-only rescaling of feed-forward network (FFN) activations to repair tool calls and other structured outputs, using multiplicative coordinate masks rather than additive steering vectors. Amplitude Gating (AG) keeps pretrained weights fixed and applies magnitude/entropy-based masks at six SwiGLU sites, with a Newton-Schulz windowed ranking variant. Across three Qwen checkpoints and six text/tool endpoints, effects are task dependent. Under frozen local scorers, Qwen3.5-9B non-NS configurations improve two function-call endpoints by +2.61/+2.67 percentage points (pp) and static ToolBench in/out-domain by +1.64/+2.82 pp; JSON gains are small. These gains survive comparing the weaker of two observed AG runs with the stronger baseline, supporting observed-run robustness rather than population significance. Scores measure lexical or answer/action-name agreement, not executed-tool success. Under the current experimental conditions, the six-task random-forest refit improves native score by +0.46 pp with a positive conditional paired interval; transfer to other settings requires further evaluation. The results support task-specific interventions; causal mechanisms, equal-budget advantages, repeated-run uncertainty, and serving cost require further evaluation. Code and experimental data: https://doi.org/10.5281/zenodo.23654706.
尚无 AI 解读，展示原文摘要