← 研踪 PaperTrailarXiv:2611.02274 · v1
Invertible Graph Aggregation Can Increase ReLU Readout Width
首次提交 2026-11-02 06:57
版本更新 2026-11-02 06:57
雷达观测 2026-11-04 23:42
摘要
An invertible graph aggregation preserves the complete feature table, but can increase the size of a classifier that reads one node at a time. For every even $N\ge4$, we construct a labeled tree with $2N$ vertices and maximum degree three. A shared one-hidden-layer rectified linear unit (ReLU) classifier needs one hidden unit on the raw scalar features and exactly $N$ units after one lazy neighborhood mean. All aggregate values are distinct, and the mixing matrix has condition number at most 8. We determine the complete error curve for heads with a free affine term. The width separation persists under scalar perturbations. Under a specified bounded uniform-noise model with one nuisance coordinate, every one-unit aggregate head has expected error at least $1/12$, while a fixed $N$-unit head has zero error. Controlled fits compare trained heads with these explicit feasible scores. On Roman-empire and Tolokers, aggregation improves fitted prediction, and retaining raw features helps at the largest tested parameter budget. Training-only diagnostics include conditional output-layer optimality bounds for eight fixed runs.
作者备注：12 pages, 4 figures. Code and reproducibility materials: https://github.com/usamihiroyasu/invertible-graph-readout-width
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-02 06:57 · 最新观测版本
Invertible Graph Aggregation Can Increase ReLU Readout Width
An invertible graph aggregation preserves the complete feature table, but can increase the size of a classifier that reads one node at a time. For every even $N\ge4$, we construct a labeled tree with $2N$ vertices and maximum degree three. A shared one-hidden-layer rectified linear unit (ReLU) classifier needs one hidden unit on the raw scalar features and exactly $N$ units after one lazy neighborhood mean. All aggregate values are distinct, and the mixing matrix has condition number at most 8. We determine the complete error curve for heads with a free affine term. The width separation persists under scalar perturbations. Under a specified bounded uniform-noise model with one nuisance coordinate, every one-unit aggregate head has expected error at least $1/12$, while a fixed $N$-unit head has zero error. Controlled fits compare trained heads with these explicit feasible scores. On Roman-empire and Tolokers, aggregation improves fitted prediction, and retaining raw features helps at the largest tested parameter budget. Training-only diagnostics include conditional output-layer optimality bounds for eight fixed runs.
查看此版本解读 →查看此版本原文 ↗