Compute the Interactions, Not the Attributes: What a Fact Layer Must Supply in the Chinese Sexagenary System
Huixue Han, Jiaying Liu
原文摘要
Language-model systems in rule-governed domains often pair the model with a deterministic fact layer, but what that layer should compute is rarely measured. We measure it in the Chinese sexagenary system underlying BaZi chart analysis, where every fact has one table-checkable value: 682 items over twelve fact categories and 160 transit--natal collision items, each posed on a ladder of rungs that differ only in how much of the computation is done for the model. Answering directly, none of eleven open-weight models gets more than 4\% of sexagenary days or 1\% of collisions right, and supplying the attributes of every pillar does not change the collisions. A complete list of computed relations is not enough either: it leaves Qwen3-32B at 1\%, with relations listed for other pairs misattributed to the queried pair, whereas a query-filtered layer from follow-up runs, stating only the queried relations pillar by pillar, gives 92\%. On chess positions, two predictions locked in advance hold: attributes do not help, and unqueried relations are again misattributed. Three proprietary reasoning models compute 68--95\% of collisions unaided, while reasoning in smaller open models overrides supplied facts. A fact layer should compute the interactions and return the ones the question asks about.
cs.CL