跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.17576 · v1

Beyond a Single Sign: BiTE for Branch-Preserving Binary Threshold Expansion

首次提交 2026-11-13 06:56
版本更新 2026-11-13 06:56
雷达观测 2026-11-17 04:58
arXiv ↗PDF ↗

摘要

Binary neural networks (BNNs) reduce model storage and replace many arithmetic operations with efficient bitwise computations, but the single-sign activation limits their representational capacity. Each preactivation direction is compared with one threshold and reduced to one of two values, discarding variation within each sign region. We propose \textbf{BiTE} (Branch-Preserving Binary Threshold Expansion), which evaluates each feature direction against KK ordered thresholds and retains the resulting binary responses as separate inputs to the following learned operator. BiTE can distinguish up to K+1K+1 ordered activation states while allowing the following operator to learn a separate transformation for each threshold response. It also enables a controllable trade-off between the number of independently constructed feature directions and the number of ordered responses extracted from each direction. On BinaryViT, BiTE improves mean final-epoch Top-1 accuracy from 74.60±0.30%74.60\pm0.30\% to 75.83±0.15%75.83\pm0.15\% on CIFAR-10 and from 42.80±0.46%42.80\pm0.46\% to 44.07±0.32%44.07\pm0.32\% on CIFAR-100, with nearly unchanged trainable parameter counts. A matched uniform-collapse control reaches 75.33±0.15%75.33\pm0.15\% on CIFAR-10, supporting the value of retaining the threshold responses separately relative to the evaluated scalar collapse. Binary VGG-16 experiments further show a favorable observed parameter--accuracy trade-off, including a configuration that achieves 0.640.64 percentage points higher mean accuracy with 21.8%21.8\% fewer parameters than conventional width scaling. Overall, these results support branch-preserving threshold expansion as an effective way to improve binary representations and the observed parameter--accuracy trade-off.

作者备注:8 pages, 5 figures

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-13 06:56 · 最新观测版本

Beyond a Single Sign: BiTE for Branch-Preserving Binary Threshold Expansion

Binary neural networks (BNNs) reduce model storage and replace many arithmetic operations with efficient bitwise computations, but the single-sign activation limits their representational capacity. Each preactivation direction is compared with one threshold and reduced to one of two values, discarding variation within each sign region. We propose \textbf{BiTE} (Branch-Preserving Binary Threshold Expansion), which evaluates each feature direction against KK ordered thresholds and retains the resulting binary responses as separate inputs to the following learned operator. BiTE can distinguish up to K+1K+1 ordered activation states while allowing the following operator to learn a separate transformation for each threshold response. It also enables a controllable trade-off between the number of independently constructed feature directions and the number of ordered responses extracted from each direction. On BinaryViT, BiTE improves mean final-epoch Top-1 accuracy from 74.60±0.30%74.60\pm0.30\% to 75.83±0.15%75.83\pm0.15\% on CIFAR-10 and from 42.80±0.46%42.80\pm0.46\% to 44.07±0.32%44.07\pm0.32\% on CIFAR-100, with nearly unchanged trainable parameter counts. A matched uniform-collapse control reaches 75.33±0.15%75.33\pm0.15\% on CIFAR-10, supporting the value of retaining the threshold responses separately relative to the evaluated scalar collapse. Binary VGG-16 experiments further show a favorable observed parameter--accuracy trade-off, including a configuration that achieves 0.640.64 percentage points higher mean accuracy with 21.8%21.8\% fewer parameters than conventional width scaling. Overall, these results support branch-preserving threshold expansion as an effective way to improve binary representations and the observed parameter--accuracy trade-off.

查看此版本解读 →查看此版本原文 ↗