AnnoHID: LLM-Assisted Annotation Framework for Low-Resource Medical Texts
Annisa Maulida Ningtyas Universitas Gadjah Mada, Guntur Budi Herwanto, Yunita Sari Universitas Gadjah Mada, Rifki Afina Putri Universitas Gadjah Mada, Filip Kovacevic Technische Universität Wien, Alaa El-Ebshihy Technische Universität Wien, Varvara Arzt, Florina Piroi Technische Universität Wien
This paper introduces AnnoHID, a semi-automated annotation framework designed for medical texts in low-resource languages. The system integrates large language models (LLMs) for pre-annotation and human validation to support efficient and consistent annotation. We demonstrate its application to Bahasa Indonesia medical social media texts from Alodokter, a medical Q A platform, for Named Entity Recognition (NER) and Medical Concept Normalization (MCN). We conducted a user study with 21 participants to demonstrate the effectiveness of AnnoHID. The results show that LLM-assisted annotation yields higher inter-annotator agreement for both NER (\kappa = 0.76) and MCN (\kappa = 0.63) and that human review improves raw LLM NER output, raising the F1 score from 0.39 to 0.45. However, LLM assistance did not reduce annotation time and may introduce normalization bias in MCN. The framework is multilingual, human-in-the-loop, and interoperable with standard medical terminologies, such as SNOMED-CT. Future work focuses on mitigating pre-annotation bias, reducing annotation overhead, and scaling evaluations to larger datasets and additional low-resource languages.
尚无 AI 解读，展示原文摘要
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations) · 2026 · 正式发表