跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2609.12099 · v2

Representation Before Weighting: Semantic Retention in Frozen 3D Foundation-Model Fusion

首次提交 2026-09-11 02:25
版本更新 2026-11-16 02:24
雷达观测 2026-11-17 17:12
arXiv ↗PDF ↗

摘要

Frozen foundation models are increasingly composed at inference time, yet a common implementation shortcut collapses each source to a Top-1 label before heterogeneous evidence can interact. We investigate this representation bottleneck in frozen two-source 3D semantic fusion, combining dense RegionPLC predictions with sparse cross-view promptable segmentation evidence. Source outputs, geometry, scene lists, vocabulary, and evaluation protocol are held fixed while the number of retained semantic alternatives is varied. Established information-theoretic results characterize hard routing as a deterministic information bottleneck and show that the total-variation error of normalized Top-kk retention equals the discarded probability mass. On 156 held-out ScanNet200 scenes, under a fixed uniform linear pool, Top-1 retention reaches 28.47 harmonic-mean (HM) IoU, Top-20 reaches 35.49, and Full retention reaches 34.87: keeping every alternative is worse than keeping twenty. Paired scene-level bootstrap comparisons show that Top-20 exceeds Top-10 by +0.289+0.289 HM (95\% CI [+0.093,+0.475][+0.093,+0.475]) and Full by +0.617+0.617 HM ([+0.204,+1.086][+0.204,+1.086]). The ordering also depends on the operator: under geometric pooling it reverses and Full retention is best, at 36.38 HM with a fit-selected exponent. Top-20 and Full differ on 9.57\% of valid points, exclusively at locations where the two sources disagree at Top-1. These changes yield 746,318 net point corrections and are associated with greater discarded dense-source tail mass. On 50 ScanNet++ validation scenes, distribution-preserving retention also outperforms Top-1, reaching up to 26.92 HM versus 23.02. Within ScanNet200 novel classes, Full favors the rarest frequency tertile, whereas Top-20 favors the most frequent tertile.

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-16 02:24 · 最新观测版本

Representation Before Weighting: Semantic Retention in Frozen 3D Foundation-Model Fusion

Frozen foundation models are increasingly composed at inference time, yet a common implementation shortcut collapses each source to a Top-1 label before heterogeneous evidence can interact. We investigate this representation bottleneck in frozen two-source 3D semantic fusion, combining dense RegionPLC predictions with sparse cross-view promptable segmentation evidence. Source outputs, geometry, scene lists, vocabulary, and evaluation protocol are held fixed while the number of retained semantic alternatives is varied. Established information-theoretic results characterize hard routing as a deterministic information bottleneck and show that the total-variation error of normalized Top-kk retention equals the discarded probability mass. On 156 held-out ScanNet200 scenes, under a fixed uniform linear pool, Top-1 retention reaches 28.47 harmonic-mean (HM) IoU, Top-20 reaches 35.49, and Full retention reaches 34.87: keeping every alternative is worse than keeping twenty. Paired scene-level bootstrap comparisons show that Top-20 exceeds Top-10 by +0.289+0.289 HM (95\% CI [+0.093,+0.475][+0.093,+0.475]) and Full by +0.617+0.617 HM ([+0.204,+1.086][+0.204,+1.086]). The ordering also depends on the operator: under geometric pooling it reverses and Full retention is best, at 36.38 HM with a fit-selected exponent. Top-20 and Full differ on 9.57\% of valid points, exclusively at locations where the two sources disagree at Top-1. These changes yield 746,318 net point corrections and are associated with greater discarded dense-source tail mass. On 50 ScanNet++ validation scenes, distribution-preserving retention also outperforms Top-1, reaching up to 26.92 HM versus 23.02. Within ScanNet200 novel classes, Full favors the rarest frequency tertile, whereas Top-20 favors the most frequent tertile.

查看此版本解读 →查看此版本原文 ↗