基于多布尔架构的高效大语言模型
Highly Efficient and Effective LLMs with Multi-Boolean Architectures
Ba-Hien Tran, Van Minh Nguyen
AI 速读 基于摘要
提出用多核布尔参数表示大语言模型，并首次实现直接在布尔域微调，从而在微调与推理阶段大幅降低复杂度。
stat.ML · cs.LG
查看原文摘要
Weight binarization has emerged as a promising strategy to reduce the complexity of large language models (LLMs). Existing approaches fall into post-training binarization, which is simple but causes severe performance loss, and training-aware methods, which depend on full-precision latent weights, adding complexity and limiting efficiency. We propose a novel framework that represents LLMs with multi-kernel Boolean parameters and, for the first time, enables direct finetuning LMMs in the Boolean domain, eliminating the need for latent weights. This enhances representational capacity and dramatically reduces complexity during both finetuning and inference. Extensive experiments across diverse LLMs show our method outperforms recent ultra low-bit quantization and binarization techniques.