跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.20767 · v1

UNITE: Consolidating Embodied VLM Specialists via Shared-Anchor One-Bit Factorization

首次提交 2026-11-16 17:19
版本更新 2026-11-16 17:19
雷达观测 2026-11-17 17:11
arXiv ↗PDF ↗

摘要

Embodied capabilities such as visual grounding, spatial reasoning, navigation, and driving understanding are often developed in separate vision-language specialists fine-tuned from a common pretrained model.Deploying the specialists separately multiplies storage, whereas merging them into one dense model forces conflicting task updates to share parameters and loses part of each specialist's expertise.We introduce UNITE, a framework that consolidates such specialists into a compact shared-backbone mixture of experts. UNITE keeps the feed-forward block of each specialist as a routable expert and represents the expert bank through a shared-anchor one-bit factorization: a shared floating-point anchor plus expert-specific, row-scaled binary deviations.The anchor is optimized jointly with the scales and signs through closed-form alternating updates, using only source weights and no calibration data.Router adaptation then adjusts token-level expert selection with the backbone and experts frozen, updating fewer than 0.60.6M parameters (<0.01%<0.01\% of the 8.88.8B active parameters). Built from four Qwen3-VL-8B specialists, the complete UNITE model occupies 20.320.3 GB, 1.16×1.16\times the storage of one dense specialist and 59.6%59.6\% below the uncompressed MoE. Across 2222 benchmarks and R2R Val-Unseen, UNITE matches six static merging methods in mean retention of best-specialist performance, leads all of them on Scan2Cap, ScanQA, and all four navigation metrics, and surpasses every source specialist on all five 3D scene understanding benchmarks (up to +19.1+19.1 on Scan2Cap) and on R2R navigation. Relative to the uncompressed MoE, factorization is nearly lossless on 2121 of 2222 benchmarks; the exception is VSI-Bench, where frame-count experiments suggest impaired use of cross-frame evidence.UNITE thus consolidates complementary embodied capabilities near single-model storage.

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-16 17:19 · 最新观测版本

UNITE: Consolidating Embodied VLM Specialists via Shared-Anchor One-Bit Factorization

Embodied capabilities such as visual grounding, spatial reasoning, navigation, and driving understanding are often developed in separate vision-language specialists fine-tuned from a common pretrained model.Deploying the specialists separately multiplies storage, whereas merging them into one dense model forces conflicting task updates to share parameters and loses part of each specialist's expertise.We introduce UNITE, a framework that consolidates such specialists into a compact shared-backbone mixture of experts. UNITE keeps the feed-forward block of each specialist as a routable expert and represents the expert bank through a shared-anchor one-bit factorization: a shared floating-point anchor plus expert-specific, row-scaled binary deviations.The anchor is optimized jointly with the scales and signs through closed-form alternating updates, using only source weights and no calibration data.Router adaptation then adjusts token-level expert selection with the backbone and experts frozen, updating fewer than 0.60.6M parameters (<0.01%<0.01\% of the 8.88.8B active parameters). Built from four Qwen3-VL-8B specialists, the complete UNITE model occupies 20.320.3 GB, 1.16×1.16\times the storage of one dense specialist and 59.6%59.6\% below the uncompressed MoE. Across 2222 benchmarks and R2R Val-Unseen, UNITE matches six static merging methods in mean retention of best-specialist performance, leads all of them on Scan2Cap, ScanQA, and all four navigation metrics, and surpasses every source specialist on all five 3D scene understanding benchmarks (up to +19.1+19.1 on Scan2Cap) and on R2R navigation. Relative to the uncompressed MoE, factorization is nearly lossless on 2121 of 2222 benchmarks; the exception is VSI-Bench, where frame-count experiments suggest impaired use of cross-frame evidence.UNITE thus consolidates complementary embodied capabilities near single-model storage.

查看此版本解读 →查看此版本原文 ↗