← 研踪 PaperTrailarXiv:2603.28764 · v2
Geometry-aware similarity metrics for neural representations on Riemannian and statistical manifolds
首次提交 2026-03-31 01:59
版本更新 2026-11-07 20:15
雷达观测 2026-11-10 11:14
摘要
Similarity measures such as centred kernel alignment (CKA) or representational similarity analysis (RSA) are widely used to compare the representational geometries employed by different neural networks to solve the same task. Yet, because existing methods compare the extrinsic geometry of representations in state space, rather than their intrinsic geometry, they may fail to capture subtle yet crucial distinctions between fundamentally different computations. Here, we introduce metric similarity analysis (MSA), a novel method which leverages tools from Riemannian geometry to compare the intrinsic geometry of neural representations under the manifold hypothesis. We show that MSA can be used to (i) disentangle neural computations of deep networks in rich vs. lazy learning regimes, (ii) compare the dynamics of recurrent neural networks and state-space models, and (iii) investigate the statistical manifolds of diffusion models. Overall, we introduce a mathematically grounded and broadly applicable framework to understand the mechanisms behind neural computations by contrasting their intrinsic geometries.
作者备注：Accepted at NeurIPS 2026 (Oral)
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-07 20:15 · 最新观测版本
Geometry-aware similarity metrics for neural representations on Riemannian and statistical manifolds
Similarity measures such as centred kernel alignment (CKA) or representational similarity analysis (RSA) are widely used to compare the representational geometries employed by different neural networks to solve the same task. Yet, because existing methods compare the extrinsic geometry of representations in state space, rather than their intrinsic geometry, they may fail to capture subtle yet crucial distinctions between fundamentally different computations. Here, we introduce metric similarity analysis (MSA), a novel method which leverages tools from Riemannian geometry to compare the intrinsic geometry of neural representations under the manifold hypothesis. We show that MSA can be used to (i) disentangle neural computations of deep networks in rich vs. lazy learning regimes, (ii) compare the dynamics of recurrent neural networks and state-space models, and (iii) investigate the statistical manifolds of diffusion models. Overall, we introduce a mathematically grounded and broadly applicable framework to understand the mechanisms behind neural computations by contrasting their intrinsic geometries.
查看此版本解读 →查看此版本原文 ↗