前向前向算法中优良性函数的选择性与形状设计
Selectivity and Shape in the Design of Forward-Forward Goodness Functions
Talha Ruzgar Akkus, Suayp Talha Kocabay, Kamer Ali Yuksel, Hassan Sawaf
AI 速读 基于摘要
系统研究前向前向(Forward-Forward)算法中优良性函数的选择性与活动形状如何影响逐层学习,并提出基于超额峰度的 burstiness 目标。
cs.LG · cs.AI · cs.NE
查看原文摘要
The Forward-Forward (FF) algorithm trains networks layer-by-layer using a local "goodness function," yet the choice of activity statistic remains an important design question. We systematically study how selectivity and activity shape affect FF learning, introducing selective goodness functions based on top-$k$ and entmax-weighted energy alongside shape-sensitive objectives based on normalized higher-order moments. In particular, we propose burstiness goodness, a parameter-free, scale-invariant excess-kurtosis objective that rewards heavy-tailed activity patterns. Our study examines goodness functions together with activation and label-injection choices across six small-image datasets, and uses controlled experiments to isolate their effects. In validation-tuned, five-seed Fashion-MNIST experiments with matched ReLU activation, standard label injection, and per-layer scoring, burstiness improves accuracy from $73.2\pm0.9\%$ to $88.6\pm0.4\%$ over sum-of-squares. On CIFAR-10, burstiness improves block-local ResNet-18 accuracy from $28.9\pm0.5\%$ to $46.8\pm2.5\%$ under the same separate per-block label pathway. These results show that activity-shape statistics can provide effective local learning signals across the tested settings. Code is available at https://github.com/suayptalha/ff-selectivity-shape.