Francesco Ortu, Joeun Yook, Punya Syon Pandey, Bernhard Schölkopf, Alberto Cazzaniga, Rada Mihalcea, Zhijing Jin
Large language models (LLMs) are increasingly used to disseminate information, yet their handling of contested historical events and politically charged narratives remains understudied. We introduce HistoricalMisinfo, a dataset of 500 contested events from 45 countries and regions, each paired with a factual and a documented revisionist reference narrative, instantiated across 11 prompt scenarios reflecting common real-world usage. Using an LLM-as-a-judge protocol, we evaluate multiple model architectures under neutral prompts and robustness prompts that explicitly request revisionist content. Under neutral prompts, models generally align with factual references, though alignment varies by scenario and model. Robustness prompting reveals a consistent and sharp increase in revisionism scores across all models, indicating limited resistance to user-driven revisionist framing. HistoricalMisinfo offers a practical benchmark for evaluating robustness to historical revisionism in AI systems. Code and dataset: https://github.com/francescortu/PreservingHistoricalTruth
Francesco Ortu University of Trieste and Area Science Park, Zhijing Jin EuroSafeAI and University of Toronto & Vector Institute, Diego Doimo Area Science Park, Alberto Cazzaniga AREA Science Park
Vision-language models (VLMs) increasingly combine visual and textual information to perform complex tasks. However, conflicts between their internal knowledge and external visual input can lead to hallucinations and unreliable predictions. In this work, we investigate the mechanisms that VLMs use to resolve cross-modal conflicts by introducing WHOOPS-AHA!, a dataset of multimodal counterfactual queries that deliberately contradict internal commonsense knowledge. Through logit inspection, we identify a small set of attention heads that mediate this conflict. By intervening in these heads, we can steer the model towards its internal parametric knowledge or the visual information. Our results show that attention patterns on these heads effectively locate image regions that influence visual overrides, providing a more precise attribution compared to gradient-based methods.
尚无 AI 解读，展示原文摘要
cs.CL · cs.CV
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表