← 研踪 PaperTrailarXiv:2605.09294 · v3
Understanding, Predicting, and Steering LLM Behavior via Representational Effective Theory
首次提交 2026-05-10 11:42
版本更新 2026-11-01 05:19
雷达观测 2026-11-04 23:43
摘要
Large language models (LLMs) assist with writing, teaching, programming and decision-making, yet the computations underlying their behaviour remain largely opaque. Full access to every weight and activation does not by itself yield understanding: a microscopic description need not reveal how computation is organized at the level that governs behaviour. Complex systems are often understood instead through a few macroscopic variables whose dynamics capture the behaviour of interest, as temperature and pressure do for molecular motion in thermodynamics. Here we show that such an effective description of LLM computation can be learned directly from data. We introduce Representational Effective Theory (RET), a framework that learns low-dimensional macrostates and their dynamics from a model's hidden-state trajectories. These macrostates exhibit approximately closed dynamics and trace interpretable trajectories through reasoning. They also predict sycophancy and alignment faking early in generation, while steering the model towards selected macrostates produces corresponding changes in its reasoning. Together, these results show that LLM computation admits useful macroscopic descriptions that support interpretation, prediction and control.
作者备注：Project webpage: https://ustaomeroglu.github.io/RET/
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v3 · 2026-11-01 05:19 · 最新观测版本
Understanding, Predicting, and Steering LLM Behavior via Representational Effective Theory
Large language models (LLMs) assist with writing, teaching, programming and decision-making, yet the computations underlying their behaviour remain largely opaque. Full access to every weight and activation does not by itself yield understanding: a microscopic description need not reveal how computation is organized at the level that governs behaviour. Complex systems are often understood instead through a few macroscopic variables whose dynamics capture the behaviour of interest, as temperature and pressure do for molecular motion in thermodynamics. Here we show that such an effective description of LLM computation can be learned directly from data. We introduce Representational Effective Theory (RET), a framework that learns low-dimensional macrostates and their dynamics from a model's hidden-state trajectories. These macrostates exhibit approximately closed dynamics and trace interpretable trajectories through reasoning. They also predict sycophancy and alignment faking early in generation, while steering the model towards selected macrostates produces corresponding changes in its reasoning. Together, these results show that LLM computation admits useful macroscopic descriptions that support interpretation, prediction and control.
查看此版本解读 →查看此版本原文 ↗