Attention Geometry
Julio Candanedo
原文摘要
Attention normalizes learned scores into a Markov operator. In the continuum, the local sector becomes diffusion on a learned Riemannian manifold, but attention natively bonds distant points, creating nonlocal structure such that no tensor geometry can describe. The Bakry-Emery calculus, built directly from the transition operator, extends geometry to these potentially stratified spaces, giving a coordinate-free account of what vision and language transformers learn.
cs.LG