理解Transformer在学习潜在结构中的阶段性动态
Understanding the Staged Dynamics of Transformers in Learning Latent Structure
Rohan Saha, Farzane Aminmansour, Alona Fyshe
AI 速览：作者在Alchemy受控基准上训练小型decoder-only Transformer，报告模型以离散阶段、由粗到细的方式习得数据的潜在结构组件。
Language models have proven to be capable of learning the intricate latent structures in language. However, the dynamics of how such models acquire the components of that structure remain poorly understood, leaving them open to the criticism that models just remix training data. In this work, we use the Alchemy benchmark as a controlled setting in which we investigate how models acquire the different latent structure components of the data. We train a small decoder-only transformer on three controlled task variants: 1) inferring missing steps from partial contextual information, 2) composing simple rules to solve multi-step sequences, and 3) decomposing complex multi-step examples to infer intermediate states. By factorizing each task into interpretable components, we show that the model learns the different latent structure components in discrete $\textit{stages}$ and in a $\textit{coarse-to-fine}$ manner. Using inference-time ablations, we demonstrate that when certain latent structures are masked or replaced, the model can use an alternative intermediate latent structure to solve the task. We also observe an asymmetry: the model composes single step transitions robustly, but struggles to decompose complex examples to discover the atomic steps. Our findings provide insight into $\textit{how}$ a transformer model acquires latent structure, offering a detailed view of how capabilities evolve during training.