Mixture of States: Routing Token-Level Dynamics for Multimodal Generation
Liu, Haozhe, Liu, Ding, Zhuge, Mingchen, Zhou, Zijian, Xie, Tian, He, Sen, Yang, Yukang, Liu, Shuming 等 19 位作者
We introduce MoS (Mixture of States), a novel fusion paradigm for multimodal diffusion models that merges modalities using flexible, state-based interactions. The core of MoS is a learnable, token-wise router that creates denoising timestep- and input-dependent interactions between modalities' hidden states, precisely aligning token-level features with the diffusion trajectory. This router sparsely selects the top-k hidden states and is trained with an epsilon-greedy strategy, efficiently selecting contextual features with minimal learnable parameters and negligible computational overhead. We validate our design with text-to-image generation (MoS-Image) and editing (MoS-Editing), which achieve state-of-the-art results. With only 3B to 5B parameters, our models match or surpass counterparts up to 4xlarger. These findings establish MoS as a flexible and compute-efficient paradigm for scaling multimodal diffusion models. The project homepage is available at https://haozheliu-st.github.io/mos-homepage/
尚无 AI 解读，展示原文摘要
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition · 2026 · 正式发表