← 研踪 PaperTrailACL Anthology · ACL 2026:acl:2026.acl-srw.99 · 记录 #1
Constructing a Japanese Verdict Prediction Dataset for Fact-Checking of LLM-Generated Texts
来源日期（仅到月份） 2026-07
来源记录日期 2026-07
雷达观测 2026-11-01 21:44
摘要
The development of fact-checking systems for verifying the factuality of text generated by large language models (LLMs) has been advancing.In the verdict prediction step of such systems, the system determines whether claims in the generated text are supported by retrieved evidence, formulated as a natural language inference (NLI) task.This study extends the label set for verdict prediction to capture claim-evidence relationships that humans would commonly interpret as supported or refuted, even in the absence of strict logical entailment or contradiction.It also constructs a Japanese dataset comprising 28,147 instances from two sources based on this extended label set.We analyze the causes of annotation disagreement and find that ambiguity in the boundary of acceptable inference, interpretive characteristics of negative cases, and incomplete information in the evidence affect annotation variability.Using this dataset, we evaluate the performance of prompt-based verdict prediction methods and show that prompts that explicitly elicit chain-of-thought reasoning improve F1 by 4 percentage points compared to baseline.
论文发现出处
这些页面提供论文线索，不代表同行评审或质量认证。
观测到的版本
平台记录号代表本站观测到的元数据变化，不是作者修订号。来源日期仅到月份或年份时，按该月或该年首日存档。
记录 #1 · 2026-07 · 最新观测版本
Constructing a Japanese Verdict Prediction Dataset for Fact-Checking of LLM-Generated Texts
The development of fact-checking systems for verifying the factuality of text generated by large language models (LLMs) has been advancing.In the verdict prediction step of such systems, the system determines whether claims in the generated text are supported by retrieved evidence, formulated as a natural language inference (NLI) task.This study extends the label set for verdict prediction to capture claim-evidence relationships that humans would commonly interpret as supported or refuted, even in the absence of strict logical entailment or contradiction.It also constructs a Japanese dataset comprising 28,147 instances from two sources based on this extended label set.We analyze the causes of annotation disagreement and find that ambiguity in the boundary of acceptable inference, interpretive characteristics of negative cases, and incomplete information in the evidence affect annotation variability.Using this dataset, we evaluate the performance of prompt-based verdict prediction methods and show that prompts that explicitly elicit chain-of-thought reasoning improve F1 by 4 percentage points compared to baseline.
查看此版本解读 →查看此版本原文 ↗