Leonardo Ranaldi, Federico Ranaldi University of Roma "Tor Vergata"
原文摘要
Debate has emerged as a promising oversight mechanism for Large Language Models (LLMs) amid rising systemic complexity, particularly where models outperform human evaluators. Yet, Debate provides little verifiable evidence for its final judgments, and its scalability remains largely unexplored. To make oversight grounded and scale as capabilities extend, we introduce an Agentic Oversight framework. By using Dialectic Argumentation as a reasoning function, we extend this paradigm to multilingual and multimodal spaces. We employ a weak-to-strong oversight approach based on two expert models that evaluate and defend contesting answers, while a third blind judge determines the winner using Dialectic Argumentation. Experts argue only for belief-consistent answers, founding the Debate on disagreements. We experimented with six tasks on our framework in both multilingual and multimodal scenarios, and dialectic argumentation consistently outperforms single-expert baselines. Moreover, we show that dialectic judgements from a weaker model deliver argument-mediated supervision that, via fine-tuning, instils unsupervised reasoning signals in expert models.
cs.CL · cs.AI · cs.CV
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表
Federico Ranaldi University of Roma "Tor Vergata", Leonardo Ranaldi, Fabio Massimo Zanzotto University of Rome Tor Vergata, Shay B Cohen University of Edinburgh
原文摘要
LLMs often mistake what sounds true for what is formally valid. This limitation is especially evident in syllogistic reasoning, where plausible arguments can lead models to endorse conclusions that are logically invalid, a phenomenon known as Content Effect (CE).We present Boethius, a schema-guided framework for syllogistic reasoning that disentangles semantic plausibility from logical validity. Boethius adopts an auditable, quasi-formal reasoning process with two complementary stages: a Schema Module, which deduces the underlying logical form by analysing the formal structure of the premises, and an Instantiation Module, which instantiates this form over the concrete argument and evaluates validity independently of content-level semantics.Our results show that Boethius consistently outperforms existing approaches, improving syllogistic reasoning accuracy while substantially reducing CE. These gains hold for both large models in a pure in-context learning setting and smaller models trained via schema-guided trajectories using supervised fine-tuning and optimisation-based refinement.
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表
AI agents powered by LLMs plan, reason and use tools well, but remain brittle to adapt and evolve because they fail to abstract the mechanisms underlying problem-solving. Reasoning, memory, and decision-making remain fragmented, leaving the systematic reuse of experience for adaptation at scale out of reach. In this paper, we introduce EvA (Evolving Agents), a framework to improve evolution in agentic reasoning that leverages quasi-symbolic abstractions—semi-structured dynamic representations that furnish constructs to distil and reuse reasoning mechanisms. They are learned and refined through experience, and when instantiated, deliver meta-states for organising reasoning. EvA orchestrates this mechanism via a Perceptor modelling observation and action, an Actor conditioning its policy on these abstractions, and a Controller overseeing structure to explore paths or initiate rollbacks. As experience accumulates, EvA refines both its abstractions and the modules that construct and instantiate them by learning past trajectories into reusable mechanisms. Our initial analysis shows that EvA improves accuracy and adapts under mid-episode distribution shifts that cause agents to plateau on complex reasoning and interactive planning tasks. These results position EvA as a step towards adaptive reasoning, memory, and meta-control.
cs.CL · cs.AI
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表