← 研踪 PaperTrailACL Anthology · ACL 2026:acl:2026.acl-long.1799 · 记录 #1
BEFT: Bias-Efficient Fine-Tuning of Language Models in Low-Data Regimes
来源日期（仅到月份） 2026-07
来源记录日期 2026-07
雷达观测 2026-11-01 21:48
摘要
Fine-tuning the bias terms of large language models (LLMs) has the potential to achieve unprecedented parameter efficiency while maintaining competitive performance, particularly in low-data regimes. However, the link between fine-tuning different bias terms (i.e., \boldsymbol{b}_q, \boldsymbol{b}_k, \boldsymbol{b}_v in the query, key, or value projections) and downstream performance remains largely unclear to date. In this paper, we investigate the link between fine-tuning \boldsymbol{b}_q, \boldsymbol{b}_k, \boldsymbol{b}_v with the performance of the downstream task. Our key finding is that directly fine-tuning \boldsymbol{b}_v generally leads to higher downstream performance in low-data regimes, in comparison to \boldsymbol{b}_q and \boldsymbol{b}_k. We extensively evaluate this unique property across a wide range of LLMs spanning encoder-only and decoder-only architectures up to 6.7B parameters (including bias-free LLMs). Our results provide strong evidence for the effectiveness of directly fine-tuning \boldsymbol{b}_v across various downstream tasks. The implementation code is available at https://github.com/whubaichuan/BEFT.
论文发现出处
这些页面提供论文线索，不代表同行评审或质量认证。
观测到的版本
平台记录号代表本站观测到的元数据变化，不是作者修订号。来源日期仅到月份或年份时，按该月或该年首日存档。
记录 #1 · 2026-07 · 最新观测版本
BEFT: Bias-Efficient Fine-Tuning of Language Models in Low-Data Regimes
Fine-tuning the bias terms of large language models (LLMs) has the potential to achieve unprecedented parameter efficiency while maintaining competitive performance, particularly in low-data regimes. However, the link between fine-tuning different bias terms (i.e., \boldsymbol{b}_q, \boldsymbol{b}_k, \boldsymbol{b}_v in the query, key, or value projections) and downstream performance remains largely unclear to date. In this paper, we investigate the link between fine-tuning \boldsymbol{b}_q, \boldsymbol{b}_k, \boldsymbol{b}_v with the performance of the downstream task. Our key finding is that directly fine-tuning \boldsymbol{b}_v generally leads to higher downstream performance in low-data regimes, in comparison to \boldsymbol{b}_q and \boldsymbol{b}_k. We extensively evaluate this unique property across a wide range of LLMs spanning encoder-only and decoder-only architectures up to 6.7B parameters (including bias-free LLMs). Our results provide strong evidence for the effectiveness of directly fine-tuning \boldsymbol{b}_v across various downstream tasks. The implementation code is available at https://github.com/whubaichuan/BEFT.
查看此版本解读 →查看此版本原文 ↗