← 研踪 PaperTrailarXiv:2606.05109 · v2
RePercENT: Scaling Disentangled Representation Learning Beyond Two Modalities
首次提交 2026-06-04 01:10
版本更新 2026-11-02 23:18
雷达观测 2026-11-04 23:41
摘要
To leverage the full potential of multimodal data, we need representations that go beyond the state-of-the-art alignment and fusion approaches and exploit all cross-modal interactions without sacrificing modality-specific information. Learning disentangled representations is a principled way to identify the underlying shared and unique factors that are hidden in observational data. However, while multimodal disentanglement is a compelling paradigm, existing methods are largely confined to the two-modality regime due to its inherent scalability bottleneck. To address this, we propose RePercENT, a self-supervised framework designed to surpass these limitations and unlocks scalable pairwise disentanglement beyond two modalities. Through a multimodal `plug-and-play' architecture, our approach operates directly on pre-extracted embeddings, eliminating the need for extensive joint pre-training while making no assumptions regarding the underlying modalities or foundation model backbones. Moreover, we introduce a joint optimization objective for simultaneously deriving the shared and unique components, and provide formal theoretical guarantees that characterize the optimality of our solution. Spanning modalities from images and videos to clinical reports and omics, and tasks from sentiment analysis to tumor classification, RePercENT successfully recovers disentangled components while maintaining competitive performance and significantly reducing computational complexity.
作者备注：Accepted at the 40th Conference on Neural Information Processing Systems (NeurIPS 2026)
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-02 23:18 · 最新观测版本
RePercENT: Scaling Disentangled Representation Learning Beyond Two Modalities
To leverage the full potential of multimodal data, we need representations that go beyond the state-of-the-art alignment and fusion approaches and exploit all cross-modal interactions without sacrificing modality-specific information. Learning disentangled representations is a principled way to identify the underlying shared and unique factors that are hidden in observational data. However, while multimodal disentanglement is a compelling paradigm, existing methods are largely confined to the two-modality regime due to its inherent scalability bottleneck. To address this, we propose RePercENT, a self-supervised framework designed to surpass these limitations and unlocks scalable pairwise disentanglement beyond two modalities. Through a multimodal `plug-and-play' architecture, our approach operates directly on pre-extracted embeddings, eliminating the need for extensive joint pre-training while making no assumptions regarding the underlying modalities or foundation model backbones. Moreover, we introduce a joint optimization objective for simultaneously deriving the shared and unique components, and provide formal theoretical guarantees that characterize the optimality of our solution. Spanning modalities from images and videos to clinical reports and omics, and tasks from sentiment analysis to tumor classification, RePercENT successfully recovers disentangled components while maintaining competitive performance and significantly reducing computational complexity.
查看此版本解读 →查看此版本原文 ↗