On the Structure of Stationary Solutions to McKean-Vlasov Equations with Applications to Noisy Transformers
Krishnakumar Balasubramanian, Sayan Banerjee, Philippe Rigollet
We study stationary solutions of McKean-Vlasov equations on the circle. We establish an exact equivalence between solutions of the stationary McKean-Vlasov equation and an infinite-dimensional quadratic system of equations over Fourier coefficients, allowing explicit characterization of stationary states in a sequence space rather than a function space. This framework provides a transparent description of local bifurcations, their periodicity and resonance structures, while accommodating singular potentials. We derive analytic expressions for the emergence, form and shape (supercritical, critical, subcritical or transcritical) of bifurcations involving possibly multiple Fourier modes and connect them with discontinuous phase transitions. We also characterize, under suitable assumptions, the detailed structure of the stationary bifurcating solutions through leading small-amplitude coefficients for arbitrarily many fixed Fourier modes. At the global level, we establish regularity and concavity properties of the free energy landscape, proving existence, compactness, and coexistence of globally minimizing stationary measures, and identifying discontinuous phase transitions with points of non-differentiability of the minimum free energy map. We apply the theory to the Noisy Mean-Field Transformer model, where we show how changing the inverse temperature $β$ affects the geometry of the infinitely many bifurcations from the uniform measure. We also discuss how increasing $β$ motivates the study of approximate multi-mode stationary profiles as candidates for a separate metastability analysis; persistence times require additional dynamical estimates. Moreover, we determine the exact threshold $β_0\simeq2.447$ separating continuous and discontinuous phase transitions in this model. The same threshold marks the change of the first bifurcation from supercritical to subcritical.
尚无 AI 解读，展示原文摘要