对齐在哪里出现,又意味着什么?对柏拉图表示假说的进一步审视
Where Does Alignment Emerge and What Does It Mean? A Closer Look at the Platonic Representation Hypothesis
Julia Herbst, Simon Roschmann, Quentin Bouniot, Zeynep Akata
AI 速读 基于摘要
该研究考察单模态模型层间与表示维度上的对齐出现位置,检验柏拉图表示假说在新模型上的能力-对齐趋势,并用单模态与多模态基线重新解读跨模态对齐。
cs.LG
查看原文摘要
The Platonic Representation Hypothesis (PRH) suggests that neural networks are converging toward similar representation spaces. As language models become more capable, their representations are aligning with those of vision models. However, recent work has questioned whether this trend continues to hold for newer language models. Moreover, it has been shown that representation similarity metrics can be confounded by model depth and width, requiring appropriate baselines for interpretation. In this work, we first investigate where alignment emerges between unimodal models across model layers and representation dimensions. Next, we find that the PRH capability-alignment trend persists for newer models as long as the captions are sufficiently clean and informative. Finally, we contextualize cross-modal alignment, introducing unimodal and multimodal baselines on various datasets. Our results show that the cross-modal representational alignment of unimodal models can surpass that of explicitly aligned vision-language models, but remains substantially behind the alignment observed for models of the same modality. Code is available at https://github.com/ExplainableML/prh-closer-look.