← 研踪 PaperTrailarXiv:2611.13597 · v2
Compute the Interactions, Not the Attributes: What a Fact Layer Must Supply in the Chinese Sexagenary System
首次提交 2026-11-10 13:15
版本更新 2026-11-11 16:29
雷达观测 2026-11-13 11:11
摘要
Language-model systems in rule-governed domains often pair the model with a deterministic fact layer, but what that layer should compute is rarely measured. We measure it in the Chinese sexagenary system underlying BaZi chart analysis, where every fact has one table-checkable value: 682 items over twelve fact categories and 160 transit--natal collision items, each posed on a ladder of rungs that differ only in how much of the computation is done for the model. Answering directly, none of eleven open-weight models gets more than 4\% of sexagenary days or 1\% of collisions right, and supplying the attributes of every pillar does not change the collisions. A complete list of computed relations is not enough either: it leaves Qwen3-32B at 1\%, with relations listed for other pairs misattributed to the queried pair, whereas a query-filtered layer from follow-up runs, stating only the queried relations pillar by pillar, gives 92\%. On chess positions, two predictions locked in advance hold: attributes do not help, and unqueried relations are again misattributed. Three proprietary reasoning models compute 68--95\% of collisions unaided, while reasoning in smaller open models overrides supplied facts. A fact layer should compute the interactions and return the ones the question asks about.
作者备注：v2: abstract, introduction, results framing and discussion revised; claims scoped to the evidence; no change to data, numbers or experiments. 26 pages (8 main), 6 figures, 17 tables
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-11 16:29 · 最新观测版本
Compute the Interactions, Not the Attributes: What a Fact Layer Must Supply in the Chinese Sexagenary System
Language-model systems in rule-governed domains often pair the model with a deterministic fact layer, but what that layer should compute is rarely measured. We measure it in the Chinese sexagenary system underlying BaZi chart analysis, where every fact has one table-checkable value: 682 items over twelve fact categories and 160 transit--natal collision items, each posed on a ladder of rungs that differ only in how much of the computation is done for the model. Answering directly, none of eleven open-weight models gets more than 4\% of sexagenary days or 1\% of collisions right, and supplying the attributes of every pillar does not change the collisions. A complete list of computed relations is not enough either: it leaves Qwen3-32B at 1\%, with relations listed for other pairs misattributed to the queried pair, whereas a query-filtered layer from follow-up runs, stating only the queried relations pillar by pillar, gives 92\%. On chess positions, two predictions locked in advance hold: attributes do not help, and unqueried relations are again misattributed. Three proprietary reasoning models compute 68--95\% of collisions unaided, while reasoning in smaller open models overrides supplied facts. A fact layer should compute the interactions and return the ones the question asks about.
查看此版本解读 →查看此版本原文 ↗v1 · 2026-11-10 13:15
Compute the Interactions, Not the Attributes: What a Fact Layer Must Supply in the Chinese Sexagenary System
Language-model systems in rule-governed domains often pair the model with a deterministic fact layer, but which facts that layer should supply is rarely measured. We measure it in the Chinese sexagenary system underlying BaZi chart analysis, where every fact has one table-checkable value: 682 items over twelve fact categories and 160 transit--natal collision items, posed closed-book, with the reference table, and with computed facts at increasing levels. In eleven open-weight models answering directly, errors concentrate where attributes must be combined: none gets more than 4\% of sexagenary days or 1\% of collisions right. Static attributes have no consistent effect, and the complete relation list leaves most direct-answering models near zero although it contains the answer. A query-filtered layer (exploratory) raises Qwen3-8B and Qwen3-32B to 78--92\% and four further models to 96--100\%. Three proprietary reasoning models compute 68--95\% of collisions unaided. On chess positions, under predictions locked in advance, attributes again do not substitute for relations and listed distractors are misattributed. The fact layer must supply computed interactions; how they are presented depends on the model and the domain.
查看此版本解读 →查看此版本原文 ↗