← 研踪 PaperTrailarXiv:2510.25069 · v3
TOPol: Reconstructing and Interpreting Multidimensional Semantic Polarity Fields
首次提交 2025-10-29 09:14
版本更新 2026-11-01 04:28
雷达观测 2026-11-04 23:43
摘要
Polarity in natural language processing has long been operationalized as a single axis: the sentiment construct. We challenge this one-dimensional view by asking what a topic's documents are bipolar about, without specifying an answer in advance. TOPol inverts the usual concept-direction paradigm: rather than mapping a named construct to one direction, it maps a topic to the full set of statistically reproducible bipolar semantic axes its documents support in a transformer's residual stream, and asks only afterwards which of them correspond to a known construct. These axes need not be evaluative. The method is a recursive, orthogonal-by-construction bipartition of each topic's document cloud, with a candidate axis promoted to a finding only if it survives one pre-specified split-half stability gate. We calibrate against a known answer first: on 9,986 Amazon reviews (14 discovered topics) in Gemma 3 4B, the star-rating difference-in-means vector serves as a ruler known by construction to be sentiment. Discovery blindly re-derives the sentiment axis in every topic (cosine 0.71-0.97) without ever seeing the rating label, while the same gate certifies 543 axes, of which 106 are near-orthogonal to sentiment yet equally reproducible; each topic supports 33-45 mutually orthogonal axes, and sentiment axes resolve in later layers than the rest. Differential vocabulary, an internals-grounded natural-language decoder and vocabulary-based clustering agree on what the non-sentiment axes contrast; clustering by shared vocabulary surfaces 99 families, 9 recurring across topics. Direction similarity to the ruler predicts an independent sentiment estimate only weakly ($r=0.243$). Polarity is better treated as a property of a topic's representational geometry, with a measurable rank and a per-axis depth, than as one universal scalar.
作者备注：16 pages, 8 figures and 5 tables
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v3 · 2026-11-01 04:28 · 最新观测版本
TOPol: Reconstructing and Interpreting Multidimensional Semantic Polarity Fields
Polarity in natural language processing has long been operationalized as a single axis: the sentiment construct. We challenge this one-dimensional view by asking what a topic's documents are bipolar about, without specifying an answer in advance. TOPol inverts the usual concept-direction paradigm: rather than mapping a named construct to one direction, it maps a topic to the full set of statistically reproducible bipolar semantic axes its documents support in a transformer's residual stream, and asks only afterwards which of them correspond to a known construct. These axes need not be evaluative. The method is a recursive, orthogonal-by-construction bipartition of each topic's document cloud, with a candidate axis promoted to a finding only if it survives one pre-specified split-half stability gate. We calibrate against a known answer first: on 9,986 Amazon reviews (14 discovered topics) in Gemma 3 4B, the star-rating difference-in-means vector serves as a ruler known by construction to be sentiment. Discovery blindly re-derives the sentiment axis in every topic (cosine 0.71-0.97) without ever seeing the rating label, while the same gate certifies 543 axes, of which 106 are near-orthogonal to sentiment yet equally reproducible; each topic supports 33-45 mutually orthogonal axes, and sentiment axes resolve in later layers than the rest. Differential vocabulary, an internals-grounded natural-language decoder and vocabulary-based clustering agree on what the non-sentiment axes contrast; clustering by shared vocabulary surfaces 99 families, 9 recurring across topics. Direction similarity to the ruler predicts an independent sentiment estimate only weakly ($r=0.243$). Polarity is better treated as a property of a topic's representational geometry, with a measurable rank and a per-axis depth, than as one universal scalar.
查看此版本解读 →查看此版本原文 ↗