← 研踪 PaperTrailarXiv:2609.12099 · v2
Representation Before Weighting: Semantic Retention in Frozen 3D Foundation-Model Fusion
首次提交 2026-09-11 02:25
版本更新 2026-11-16 02:24
雷达观测 2026-11-17 17:12
摘要
Frozen foundation models are increasingly composed at inference time, yet a common implementation shortcut collapses each source to a Top-1 label before heterogeneous evidence can interact. We investigate this representation bottleneck in frozen two-source 3D semantic fusion, combining dense RegionPLC predictions with sparse cross-view promptable segmentation evidence. Source outputs, geometry, scene lists, vocabulary, and evaluation protocol are held fixed while the number of retained semantic alternatives is varied. Established information-theoretic results characterize hard routing as a deterministic information bottleneck and show that the total-variation error of normalized Top- retention equals the discarded probability mass. On 156 held-out ScanNet200 scenes, under a fixed uniform linear pool, Top-1 retention reaches 28.47 harmonic-mean (HM) IoU, Top-20 reaches 35.49, and Full retention reaches 34.87: keeping every alternative is worse than keeping twenty. Paired scene-level bootstrap comparisons show that Top-20 exceeds Top-10 by HM (95\% CI ) and Full by HM (). The ordering also depends on the operator: under geometric pooling it reverses and Full retention is best, at 36.38 HM with a fit-selected exponent. Top-20 and Full differ on 9.57\% of valid points, exclusively at locations where the two sources disagree at Top-1. These changes yield 746,318 net point corrections and are associated with greater discarded dense-source tail mass. On 50 ScanNet++ validation scenes, distribution-preserving retention also outperforms Top-1, reaching up to 26.92 HM versus 23.02. Within ScanNet200 novel classes, Full favors the rarest frequency tertile, whereas Top-20 favors the most frequent tertile.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-16 02:24 · 最新观测版本
Representation Before Weighting: Semantic Retention in Frozen 3D Foundation-Model Fusion
Frozen foundation models are increasingly composed at inference time, yet a common implementation shortcut collapses each source to a Top-1 label before heterogeneous evidence can interact. We investigate this representation bottleneck in frozen two-source 3D semantic fusion, combining dense RegionPLC predictions with sparse cross-view promptable segmentation evidence. Source outputs, geometry, scene lists, vocabulary, and evaluation protocol are held fixed while the number of retained semantic alternatives is varied. Established information-theoretic results characterize hard routing as a deterministic information bottleneck and show that the total-variation error of normalized Top- retention equals the discarded probability mass. On 156 held-out ScanNet200 scenes, under a fixed uniform linear pool, Top-1 retention reaches 28.47 harmonic-mean (HM) IoU, Top-20 reaches 35.49, and Full retention reaches 34.87: keeping every alternative is worse than keeping twenty. Paired scene-level bootstrap comparisons show that Top-20 exceeds Top-10 by HM (95\% CI ) and Full by HM (). The ordering also depends on the operator: under geometric pooling it reverses and Full retention is best, at 36.38 HM with a fit-selected exponent. Top-20 and Full differ on 9.57\% of valid points, exclusively at locations where the two sources disagree at Top-1. These changes yield 746,318 net point corrections and are associated with greater discarded dense-source tail mass. On 50 ScanNet++ validation scenes, distribution-preserving retention also outperforms Top-1, reaching up to 26.92 HM versus 23.02. Within ScanNet200 novel classes, Full favors the rarest frequency tertile, whereas Top-20 favors the most frequent tertile.
查看此版本解读 →查看此版本原文 ↗