Factual State Discovery Benchmark: Evaluating Fact Elicitation in Polish Tax Law
Mateusz Bystroński, Kamil Tagowski Wroclaw University of Science and Technology, Denis Janiak, Julia Farganus, Lukasz Augustyniak Wroclaw University of Science and Technology, Monika Kajdanowicz Academy of Economics in Poznan, Tomasz Jan Kajdanowicz Wroclaw University of Science and Technology
原文摘要
Before a tax authority can issue a ruling, it must receive a complete description of the taxpayer’s situation—yet no benchmark measures whether language models can systematically elicit all relevant facts through dialogue.We introduce FSDBench (Factual State Discovery Benchmark), in which a discovery agent questions a simulated taxpayer grounded in a real tax document.The dataset comprises 500 narratives from official Polish tax interpretations, decomposed into 32 874 atomic facts with validated supported precision (97.6%), atomicity (93.8%), and sentence coverage (96.0%).Experiments with four models show that even the best system recovers only 77% of facts on easy samples and under 49% on hard samples after 50 turns.These findings establish conversational fact elicitation as a challenging open problem requiring retrieval-augmented and adaptive questioning strategies.
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop) · 2026 · 正式发表