← 研踪 PaperTrailarXiv:2611.17576 · v1
Beyond a Single Sign: BiTE for Branch-Preserving Binary Threshold Expansion
首次提交 2026-11-13 06:56
版本更新 2026-11-13 06:56
雷达观测 2026-11-17 04:58
摘要
Binary neural networks (BNNs) reduce model storage and replace many arithmetic operations with efficient bitwise computations, but the single-sign activation limits their representational capacity. Each preactivation direction is compared with one threshold and reduced to one of two values, discarding variation within each sign region. We propose \textbf{BiTE} (Branch-Preserving Binary Threshold Expansion), which evaluates each feature direction against ordered thresholds and retains the resulting binary responses as separate inputs to the following learned operator. BiTE can distinguish up to ordered activation states while allowing the following operator to learn a separate transformation for each threshold response. It also enables a controllable trade-off between the number of independently constructed feature directions and the number of ordered responses extracted from each direction. On BinaryViT, BiTE improves mean final-epoch Top-1 accuracy from to on CIFAR-10 and from to on CIFAR-100, with nearly unchanged trainable parameter counts. A matched uniform-collapse control reaches on CIFAR-10, supporting the value of retaining the threshold responses separately relative to the evaluated scalar collapse. Binary VGG-16 experiments further show a favorable observed parameter--accuracy trade-off, including a configuration that achieves percentage points higher mean accuracy with fewer parameters than conventional width scaling. Overall, these results support branch-preserving threshold expansion as an effective way to improve binary representations and the observed parameter--accuracy trade-off.
作者备注:8 pages, 5 figures
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-13 06:56 · 最新观测版本
Beyond a Single Sign: BiTE for Branch-Preserving Binary Threshold Expansion
Binary neural networks (BNNs) reduce model storage and replace many arithmetic operations with efficient bitwise computations, but the single-sign activation limits their representational capacity. Each preactivation direction is compared with one threshold and reduced to one of two values, discarding variation within each sign region. We propose \textbf{BiTE} (Branch-Preserving Binary Threshold Expansion), which evaluates each feature direction against ordered thresholds and retains the resulting binary responses as separate inputs to the following learned operator. BiTE can distinguish up to ordered activation states while allowing the following operator to learn a separate transformation for each threshold response. It also enables a controllable trade-off between the number of independently constructed feature directions and the number of ordered responses extracted from each direction. On BinaryViT, BiTE improves mean final-epoch Top-1 accuracy from to on CIFAR-10 and from to on CIFAR-100, with nearly unchanged trainable parameter counts. A matched uniform-collapse control reaches on CIFAR-10, supporting the value of retaining the threshold responses separately relative to the evaluated scalar collapse. Binary VGG-16 experiments further show a favorable observed parameter--accuracy trade-off, including a configuration that achieves percentage points higher mean accuracy with fewer parameters than conventional width scaling. Overall, these results support branch-preserving threshold expansion as an effective way to improve binary representations and the observed parameter--accuracy trade-off.
查看此版本解读 →查看此版本原文 ↗