← 研踪 PaperTrailarXiv:2611.18219 · v1
Large language models put existential risk from artificial intelligence at 10%
首次提交 2026-11-13 22:22
版本更新 2026-11-13 22:22
雷达观测 2026-11-16 11:41
摘要
How likely is it that advanced artificial intelligence (AI) will cause human extinction or a similarly permanent disempowerment of humanity? Humans, including AI experts, frequently disagree on this question. Yet one important perspective is missing from this debate: that of AI systems themselves. We put the question to 241 large language models (LLMs) at every available reasoning setting. The models converged on a median risk of 10.0%, with the middle 90% of their estimates between 3.5% and 15.8%, far narrower than the spread among human experts. More expensive, and typically more capable, models stated higher risks; newer models showed more uniform, but not higher, risks. More reasoning barely changed the models' estimates but widened their uncertainty, mostly toward higher risk. Almost all models named loss of control over AI as the most likely cause. These answers are not independent judgments; they likely echo training data, including published expert polls, and choices made in aligning the models. Still, they document the position that current AI systems state on the risks of their own technology. Whether this position reflects any genuine understanding of the systems' own capabilities remains open. Repeated measurements can show whether it shifts as AI systems advance, and how training and alignment shape it is a question for both research and public debate.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-13 22:22 · 最新观测版本
Large language models put existential risk from artificial intelligence at 10%
How likely is it that advanced artificial intelligence (AI) will cause human extinction or a similarly permanent disempowerment of humanity? Humans, including AI experts, frequently disagree on this question. Yet one important perspective is missing from this debate: that of AI systems themselves. We put the question to 241 large language models (LLMs) at every available reasoning setting. The models converged on a median risk of 10.0%, with the middle 90% of their estimates between 3.5% and 15.8%, far narrower than the spread among human experts. More expensive, and typically more capable, models stated higher risks; newer models showed more uniform, but not higher, risks. More reasoning barely changed the models' estimates but widened their uncertainty, mostly toward higher risk. Almost all models named loss of control over AI as the most likely cause. These answers are not independent judgments; they likely echo training data, including published expert polls, and choices made in aligning the models. Still, they document the position that current AI systems state on the risks of their own technology. Whether this position reflects any genuine understanding of the systems' own capabilities remains open. Repeated measurements can show whether it shifts as AI systems advance, and how training and alignment shape it is a question for both research and public debate.
查看此版本解读 →查看此版本原文 ↗