arXiv:2601.16936 · v2

Is BatchEnsemble a Single Model? On Calibration and Diversity of Efficient Ensembles

首次提交 2026-01-24 01:50

版本更新 2026-10-27 22:30

雷达观测 2026-10-29 00:34

摘要

In resource-constrained and low-latency settings, uncertainty estimates must be efficiently obtained. Deep Ensembles provide robust epistemic uncertainty (EU) but require training multiple full-size models. BatchEnsemble aims to deliver ensemble-like EU at far lower parameter and memory cost by applying learned rank-1 perturbations to a shared base network. We show that BatchEnsemble not only underperforms Deep Ensembles but closely tracks a single model baseline in terms of accuracy, calibration and out-of-distribution (OOD) detection on CIFAR10/10C/SVHN. A controlled study on MNIST finds members are near-identical in function and parameter space, indicating limited capacity to realize distinct predictive modes. Thus, BatchEnsemble behaves more like a single model than a true ensemble.

作者备注：Accepted at the 1st workshop on Epistemic Intelligence in Machine Learning at EurIPS 2025