← 研踪 PaperTrailarXiv:2609.01470 · v2
RadMatch: Auditable Radiology Report Evaluation via Finding-Level Matching
首次提交 2026-09-02 00:05
版本更新 2026-11-18 20:19
雷达观测 2026-11-19 15:04
摘要
As AI systems are increasingly used to draft radiology reports, reliably evaluating their clinical quality remains a critical challenge. We introduce RadMatch, a multi-stage LLM-based framework that turns report evaluation into a structured, auditable finding-level record. It extracts findings from a reference and a candidate report, matches them by clinical equivalence, grades each match over ten clinical attributes, and gives every match and unmatched finding a verdict and a clinical-significance tier. To validate metrics on CT, we build and release MetricEval-BodyCT, the first public metric-evaluation benchmark of chest and abdomen CT reports annotated by radiologists error by error. On it and three chest X-ray benchmarks, RadMatch with an open 31B judge on consumer GPUs agrees with radiologists more closely than previous published metrics, and even reaches inter-expert agreement on ReXVal and on the significant errors of MetricEval-BodyCT. Radiologists auditing its decisions on 60 studies confirm 98.5% of its matches and 93.6% of the errors it counts. When used to evaluate state-of-the-art radiology report generation models, RadMatch shows that most CT models, unlike several chest X-ray models, actually leave more significant errors than an image-blind normal template, as their added unsupported findings outnumber the reference findings they correctly report. We provide RadMatch's code, its dashboard, the MetricEval-BodyCT dataset and all per-report scores.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-18 20:19 · 最新观测版本
RadMatch: Auditable Radiology Report Evaluation via Finding-Level Matching
As AI systems are increasingly used to draft radiology reports, reliably evaluating their clinical quality remains a critical challenge. We introduce RadMatch, a multi-stage LLM-based framework that turns report evaluation into a structured, auditable finding-level record. It extracts findings from a reference and a candidate report, matches them by clinical equivalence, grades each match over ten clinical attributes, and gives every match and unmatched finding a verdict and a clinical-significance tier. To validate metrics on CT, we build and release MetricEval-BodyCT, the first public metric-evaluation benchmark of chest and abdomen CT reports annotated by radiologists error by error. On it and three chest X-ray benchmarks, RadMatch with an open 31B judge on consumer GPUs agrees with radiologists more closely than previous published metrics, and even reaches inter-expert agreement on ReXVal and on the significant errors of MetricEval-BodyCT. Radiologists auditing its decisions on 60 studies confirm 98.5% of its matches and 93.6% of the errors it counts. When used to evaluate state-of-the-art radiology report generation models, RadMatch shows that most CT models, unlike several chest X-ray models, actually leave more significant errors than an image-blind normal template, as their added unsupported findings outnumber the reference findings they correctly report. We provide RadMatch's code, its dashboard, the MetricEval-BodyCT dataset and all per-report scores.
查看此版本解读 →查看此版本原文 ↗