Displacement Geometry Captures Platonic Shared Reality Across Models and Modalities
Chenming Shang, Yujin Tang, Jun Jie Ou Yang, Ruize Xu, Adam Breuer, Nikhil Singh
原文摘要
The Platonic Representation Hypothesis (PRH) claims that independently trained models converge on a shared statistical model of reality, yet recent work finds that the evidence weakens under some conditions. In this paper, we measure what models share through the directions (displacement vectors) between samples. Under a single orthogonal alignment limited to just rotation and reflection, these displacement vectors are substantially preserved across 44 independently trained vision and language encoders spanning modalities and asymmetric capability pairs, consistent with the PRH. Pointwise similarity, however, is lower. Representations split into a shared semantic component that is linearly aligned across models, and a private component that is not, which we interpret as having model-specific capabilities. We trace this geometry to concept-level structure: within a model, parent concepts are orthogonal to their child variation vectors; across models, concept displacements are parallel. We predict (and experiments confirm) that fine-tuning preserves Pointwise similarity but reduces displacement similarity, and relational distillation does the opposite. Because semantics align linearly but capabilities do not, we propose importing capabilities from one model to another using a single cached forward pass through the source. We call this Shadow Casting. As a proof of concept, our ShadowCLIP instantiation outperforms strong fine-tuned baselines at far less compute. A cache can be released alongside open model weights, letting one model's capabilities be downloaded and imported into other models with frozen backbones.
cs.LG