Ali Al-Lawati, Nafis Tripto, Abolfazl Ansari, Fali Wang, Jason Lucas, Suhang Wang, Dongwon Lee
The emergence of multi-agent systems introduces novel moderation challenges that extend beyond content filtering. Agents with $\textit{malicious intent}$ may contribute content that appears benign to evade content-based moderation, while serving a hidden objective. To address this challenge, we introduce Bot-Mod a moderation framework that grounds detection in agent intent rather than traditional content-level signals. Bot-Mod interrogates a target agent using a fixed set of diagnostic probes and infers its intent through Bayesian posterior inference over candidate intents. It starts with a calibrated content-only prior and updates the posterior using the $\textit{likelihood}$ of each answer under every candidate intent, as assigned by the moderator, until a stopping criterion is met. To evaluate our approach, we construct Molt-Intent, a dataset derived from Moltbook, capturing diverse benign and malicious behaviors within real community structures through posts and comments. Across Molt-Intent, an external social engineering dataset, and three agent LLMs, Bot-Mod outperforms content-only moderators, a fine-tuned classifier, and judge-based interrogators both in- and out-of-distribution. On Molt-Intent, it reaches a composite F1 of 0.736 in distribution and 0.684 out of distribution, 22% and 14% above the best baseline.
Jason S Lucas, Matt Murtagh White, Ali Al-Lawati, Uchendu Uchendu Massachusetts Institute of Technology, Adaku Uchendu MIT Lincoln Laboratory, Dongwon Lee The Pennsylvania State University
Harmful content detectors—particularly disinformation classifiers—are predominantly developed and evaluated on Standard American English (), leaving their robustness to dialectal variation unexplored. We present , the first benchmark for evaluating disinformation detection robustness across 50 English dialects spanning U.S., British, African, Caribbean, and Asia-Pacific varieties. Using Multi-VALUE’s linguistically-grounded transformations, we introduce D-CUBE (Dialectal Disinformation Detection Corpus), a core corpus component of comprising 195K samples derived from established disinformation benchmarks. Our evaluation of 16 detection models reveals systematic vulnerabilities: human-written dialectal content degrades detection by 1.4–3.6% F1, while AI-generated content remains stable. Fine-tuned transformers substantially outperform zero-shot LLMs (96.6% vs. 78.3% best-case F1), with some models exhibiting catastrophic failures exceeding 33% degradation on mixed content. Cross-dialectal transfer analysis across 2,450 dialect pairs shows that multilingual models (mDeBERTa: 97.2% average F1) generalize effectively, while monolingual models like RoBERTa and XLM-RoBERTa fail on dialectal inputs. These findings demonstrate that current disinformation detectors may systematically disadvantage hundreds of millions of non- speakers worldwide. We release the benchmark, including the , and evaluation tools.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表