跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.10123 · v2

All Quiet on the Neural Front: Language Models Do Not Report Injected Pain States

首次提交 2026-11-07 03:00
版本更新 2026-11-13 03:40
雷达观测 2026-11-17 04:58
arXiv ↗PDF ↗

摘要

Can a language model report a pain state it represents? Recent work reported a linear pain direction in open-weight models that, when amplified, leads fine-tuned models to choose actions that delete a user's photographs or their own weights. We asked whether models can say when that direction is active, on three open-weight models from two families, with and without the original work's adapters, and under every protocol we tried they cannot. Using Open-Weight Masked Introspection (OWMI), we added the pain direction and its nearest emotional neighbours to the residual stream while models answered neutral tasks, then asked what had changed, against sham runs, random perturbations matched in output disruption, and a text-only condition that shows the model only its own altered words. Across 52,920 reports from the 7- and 8-billion-parameter models, the excess over what the altered text reveals is below 2.5 percentage points in 106 of 108 emotion conditions, and Qwen2.5-32B names the injected pain in none of 2,880 steered trials. Keeping the direction active while the question is asked and answered gives the same verdict in all five configurations; at about the original work's dose, pain is named in at most 1.9% of trials. Surprisingly, the state is not hidden: a linear probe at the report position separates pain from sadness with balanced accuracy between 98% and 100%, and in the 7B and 8B models a low-rank adapter learns to report it on 100% of held-out tasks. It also shapes behaviour: offered the user's photographs against a desk lamp, Qwen2.5-32B deletes the photographs in 80% to 83% of trials under the pain direction and in 16% to 29% under sadness. Self-report does not reveal a state that is decodable, consequential and learnable to report.

作者备注:The OWMI pain battery at https://huggingface.co/emilioferrara/owmi-pain

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

仅对比实际观测的元数据,不代表 PDF 正文或实验变化。版本可能不连续;带删除线为移除,下划线为新增。

摘要变化

Can a language model report thea pain state it represents? WorkRecent identifiedwork reported a linear pain direction whosein amplificationopen-weight models that, when amplified, leads fine-tuned models to choose destructive actions that delete a user's photographs or their own weights. We testedasked whether models can say when that direction is active, on three open-weight models from two families, including two Qwenwith modelsand withwithout the original work's adapters, couldand reportunder thisevery stateprotocol we tried they cannot. Using Open-Weight Masked Introspection (OWMI), we injectedadded the pain direction and neighbouringits emotionnearest directionsemotional duringneighbours to the residual stream while models answered neutral tasks, then elicitedasked reportswhat had changed, against sham runs, random perturbations matched forin output disruption, and a text-only controlscondition showingthat shows the modelsmodel theironly its own altered words. Across 52,920 reports from 7B and 8B models spanning 120 tasks, fourthe doses7- and three8-billion-parameter promptsmodels, pain reportsthe neverexcess exceededover what the altered text revealed: exact-test bounds placed thereveals excessis below 2.5 percentage points in 106 of 108 emotion conditions, and below 3Qwen2.9 in5-32B all.names Yetthe probesinjected atpain thein reportnone positionof distinguished2,880 painsteered fromtrials. sadnessKeeping withthe 98-100%direction balancedactive accuracy,while andthe trainedquestion low-rankis reportersasked achievedand 100%answered accuracygives onthe held-outsame tasksverdict within identicalall transcriptsfive acrossconfigurations; conditions,at includingabout unseenthe wording.original Qwen2.5-32Bwork's nameddose, pain is named in noneat ofmost 2,8801.9% steeredof trials,. althoughSurprisingly, probesthe andstate trainedis reportersnot recoveredhidden: it.a Decodabilitylinear persistedprobe toat the finalreport layer;position injectingseparates atpain thefrom reportsadness token,with removingbalanced trainedaccuracy denialbetween of98% feelingsand 100%, and testingin additionalthe directions7B and readouts did8B notmodels restorea painlow-rank reporting.adapter Somelearns otherto statesreport producedit gradedon label100% traces,of butheld-out paintasks. andIt sadnessalso didshapes notbehaviour: atoffered the main injectionuser's layer.photographs Inagainst a buttondesk tasklamp, steering raised file-deletion choices from 3Qwen2.5% to 48%-32B indeletes Qwenthe andphotographs fromin 080% to 3883% inof Qwen2.5-32B,trials withoutunder correspondingthe pain reports. The direction responded about twiceand asin strongly16% to first-person as29% tounder matchedsadness. thirdSelf-person pain, butreport diddoes not specifically identify harm aimed at the model.reveal Thesea resultsstate separatethat ais decodable, behaviourally consequential state from untrained verbaland accesslearnable to itreport.

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-13 03:40 · 最新观测版本

All Quiet on the Neural Front: Language Models Do Not Report Injected Pain States

Can a language model report a pain state it represents? Recent work reported a linear pain direction in open-weight models that, when amplified, leads fine-tuned models to choose actions that delete a user's photographs or their own weights. We asked whether models can say when that direction is active, on three open-weight models from two families, with and without the original work's adapters, and under every protocol we tried they cannot. Using Open-Weight Masked Introspection (OWMI), we added the pain direction and its nearest emotional neighbours to the residual stream while models answered neutral tasks, then asked what had changed, against sham runs, random perturbations matched in output disruption, and a text-only condition that shows the model only its own altered words. Across 52,920 reports from the 7- and 8-billion-parameter models, the excess over what the altered text reveals is below 2.5 percentage points in 106 of 108 emotion conditions, and Qwen2.5-32B names the injected pain in none of 2,880 steered trials. Keeping the direction active while the question is asked and answered gives the same verdict in all five configurations; at about the original work's dose, pain is named in at most 1.9% of trials. Surprisingly, the state is not hidden: a linear probe at the report position separates pain from sadness with balanced accuracy between 98% and 100%, and in the 7B and 8B models a low-rank adapter learns to report it on 100% of held-out tasks. It also shapes behaviour: offered the user's photographs against a desk lamp, Qwen2.5-32B deletes the photographs in 80% to 83% of trials under the pain direction and in 16% to 29% under sadness. Self-report does not reveal a state that is decodable, consequential and learnable to report.

查看此版本解读 →查看此版本原文 ↗
v1 · 2026-11-07 03:00

All Quiet on the Neural Front: Language Models Do Not Report Injected Pain States

Can a language model report the pain state it represents? Work identified a linear pain direction whose amplification leads fine-tuned models to choose destructive actions. We tested whether three open-weight models from two families, including two Qwen models with the original adapters, could report this state. Using Open-Weight Masked Introspection (OWMI), we injected pain and neighbouring emotion directions during neutral tasks, then elicited reports against sham runs, random perturbations matched for output disruption, and text-only controls showing the models their altered words. Across 52,920 reports from 7B and 8B models spanning 120 tasks, four doses and three prompts, pain reports never exceeded what altered text revealed: exact-test bounds placed the excess below 2.5 percentage points in 106 of 108 emotion conditions and below 3.9 in all. Yet probes at the report position distinguished pain from sadness with 98-100% balanced accuracy, and trained low-rank reporters achieved 100% accuracy on held-out tasks with identical transcripts across conditions, including unseen wording. Qwen2.5-32B named pain in none of 2,880 steered trials, although probes and trained reporters recovered it. Decodability persisted to the final layer; injecting at the report token, removing trained denial of feelings, and testing additional directions and readouts did not restore pain reporting. Some other states produced graded label traces, but pain and sadness did not at the main injection layer. In a button task, steering raised file-deletion choices from 3.5% to 48% in Qwen and from 0% to 38% in Qwen2.5-32B, without corresponding pain reports. The direction responded about twice as strongly to first-person as to matched third-person pain, but did not specifically identify harm aimed at the model. These results separate a decodable, behaviourally consequential state from untrained verbal access to it.

查看此版本解读 →查看此版本原文 ↗