Instruction-Tuned Models Locally Produce More Syntactic Overlap -- But Not More Conditional Reuse -- Than Humans Do
Zandi Eberstadt
Syntactic convergence - the tendency of speakers to adapt in language towards the grammatical profiles of their interlocutors - is a well-documented, largely implicit feature of human dialogue. Whether large language models exhibit analogous syntactic convergence towards human users relative to human baselines (including across a broad range of syntactic constructions) remains an open question. Using substitution-paradigm data from Blevins et al. [2026] in which model generations replace one speaker's turns in pre-existing human dialogues, this study measures turn-adjacent reuse of context-free grammar (CFG) rules across sixteen open-weight Llama and Gemma models (1B-70B, pretrained and instruction-tuned) at 1,901 matched positions per model. Every model showed greater CFG-rule overlap with the preceding human turn than with a sampled unrelated human prime (a context-associated effect that does not isolate syntactic priming from topical continuity), and this actual-versus-random difference was larger for lower-frequency rules. Each instruction-tuned model also showed greater natural-output overlap with the actual prime than the human response it replaced, and all eight matched architecture pairs exhibited greater actual-prime overlap after instruction tuning. However, relative to pretrained variants, instruction-tuned outputs overlapped more with unrelated primes, showed a smaller actual-versus-random increment, and had lower conditional rule-reuse odds once target rule-set size was held constant. In exploratory analyses, each model exhibited greater mean lexical and semantic similarity to the preceding turn than matched human responses did. Instruction-tuned models additionally produced responses with greater mean semantic similarity than their pretrained counterparts in all eight architecture pairs, whereas the lexical similarity results were more heterogeneous.
尚无 AI 解读，展示原文摘要