双状态槽注意力:解耦外观与身份的视频物体中心学习
Dual-State Slot Attention: Decoupling Appearance and Identity for Video Object-Centric Learning
Sieu Tran, Duc Nguyen, Hao Vo, Khoa Vo, Ngan Le
AI 速读 基于摘要
提出DSSA,将槽向量拆分为逐帧外观的局部状态与跨帧稳定的身份状态,并抑制弱匹配槽的虚假更新,以提升无监督视频物体中心学习的身份稳定性。
cs.CV
查看原文摘要
Unsupervised video object-centric learning aims to decompose dynamic scenes into persistent, object-level representations without supervision. However, existing slot-based methods struggle to maintain stable object identity in challenging settings such as rapid motion and partial occlusion. First, they typically encode both the per-frame appearance of an object and its identity across frames in a single slot vector, creating an objective conflict that leads to slot swapping: reconstruction requires sensitivity to transient visual changes, whereas temporal consistency requires invariance to them. Second, the token renormalization used in Slot Attention can amplify weakly attending slots, allowing them to absorb tokens from other objects and destabilize slot-to-object correspondence. We propose Dual-State Slot Attention (DSSA), a fully self-supervised framework that addresses these limitations by separating appearance from identity and by reducing spurious updates from weakly matching slots. DSSA decomposes each slot into a local state for per-frame appearance and an identity state for temporally stable object information, thereby aligning reconstruction and temporal consistency with separate representations. The identity state is updated through a learned recurrent transition that acts as a temporal filter on the local state, while competition-modulated aggregation (CMA) down-weights updates from weakly matching slots and prevents them from absorbing tokens from other objects. Experiments on MOVi-C, MOVi-D, and YouTube-VIS demonstrate that DSSA consistently improves segmentation quality and temporal consistency over prior methods, while also yielding stronger downstream object recognition and video dynamics prediction. Code is available at https://uark-aicv.github.io/dssa/.