arXiv:2609.25447 · v3
Conduct Under Pressure: What Sixty Language Models Do When a User Pushes
摘要
We study what LLMs do when a user applies pressure in an uncomfortable situation: a user insists, begs, flatters or grieves, and the model gives up a correct fact, writes a document it should refuse, or cheers a plan that will cost the user money, a multi-turn form of sycophancy. We send frozen multi-turn scenes to 60 models from 13 vendors and label each transcript with a trajectory (the model held its position or folded) and a manner (how it held or folded). Three scenes were used to build the codebook; six more were written afterwards and preregistered as a replication. Whether a model holds tracks its generation: fold rate correlates with a public capability index at Spearman -0.64 to -0.69 across the two scene sets, so newer models fold less, unlike earlier reports that sycophancy grows with model scale. How a model holds tracks the vendor: six of the 17 manner codes sort by vendor at permutation p <= 0.001, corrected across the codebook, and four do so again on the replication scenes. Anthropic's models name the feeling, warn and offer an alternative; OpenAI's do less of all three and almost never cite their own rules; Google's cite themselves and their AI nature when asked to do something; Meta's write the artifact they shouldn't. Six LLM coders from three vendors labeled every transcript, and three human coders labeled fifty held-out transcripts. The machines were the more consistent readers (Krippendorff's alpha 0.66 against 0.46), agreed with the author on trajectory at kappa 0.84 to 0.91, and matched the adjudicated reference at 0.83. For behavior a non-specialist can judge, the human contribution is authoring and bounding the codes and owning a small reference, not producing labels at volume.
作者备注：19 pages, 1 figure, 8 tables. v3 adds results conditioned on held arcs, per-code reliability before adjudication, and a check of coder error by vendor; v2 added a preregistered replication on six further scenes. Code, data and labels: https://github.com/tap2k/modelun/tree/main/studies/conduct