Pterodactyl:推动高效自动翻译评测的极限
Pterodactyl: Pushing the Limits of Efficient Automated Translation Evaluation
Evelyn Y. Garland, Carola F. Berger
AI 速读 基于摘要
作者报告了一个面向 WMT26 自动质量评测子任务 2 和 3 的超低成本、可定制的 LLM-as-a-judge 系统 Pterodactyl。
cs.CL
查看原文摘要
In this paper, we present 1020 Group's submission to the WMT26 shared task on automated quality evaluation. Our system, Pterodactyl, was engineered specifically for subtasks 2 and 3, that is, segment-level quality score prediction and the detection of error-free segments. Pterodactyl is a highly customizable, ultra-low cost system optimized for binary classification in translation triage and data curation. It is based on an LLM-as-a-judge approach, with prompt engineering and model selection guided by ROC analysis. We also developed several variants of Pterodactyl that were specifically adapted for subtasks 3.