Markovian Circuit Tracing: Ground-Truth Tests of Transformer State Abstractions
Abdullah X
原文摘要
Mechanistic interpretability needs tests that separate accessible information from a causally useful internal abstraction. We introduce Markovian Circuit Tracing (MCT), a controlled benchmark for testing whether transformer activations support a compact state abstraction with recoverable transition structure. We train small causal transformers on sequences from four-state hidden Markov models. The experiment fixes the latent transition law and varies emission observability. The canonical artifact contains 15 runs across five fixed seeds. Every model reaches validation loss within 0.02 nats per token of the exact Bayes predictor. Under easy observability, held-out mean rowwise Kullback-Leibler (KL) divergence from the true transition matrix is 0.114. The easy shuffled control gives 0.379. Under medium observability, recovered transition KL is 0.221. The medium shuffled control gives 0.355. Recovery collapses to control performance under hard observability. Natural activation swaps and state forcing show the same decline in causal specificity. New transition-specific controls refine this result. Untrained activations yield similar transition recovery in the easy regime. In medium, trained activations improve over the untrained control by 0.049 KL on average. Probabilistic Markov-order tests show a clear first-order gain in easy. Medium shows a smaller gain. Hard shows no meaningful gain. A second latent transition law reproduces the same observability-dependent pattern. The belief-probe control fails to support learned specificity. The fixed sparse autoencoder fails to improve transition recovery. MCT therefore tests when a state-machine interpretation earns support and when the evidence rejects it.
cs.LG