跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2510.18691 · v3

Investigating LLM Capabilities on Long Context Comprehension for Clinical Question Answering

首次提交 2025-10-21 22:50
版本更新 2026-11-13 06:58
雷达观测 2026-11-17 04:58
arXiv ↗PDF ↗

摘要

Despite the impressive reported performance of Large Language Models (LLMs) on medical Question-Answering (QA), evaluation still overrelies on multiple-choice and extractive settings primarily assessing textbook knowledge, with limited work on open-ended QA over patient records. We present a comprehensive study investigating LLM long-context (LC) comprehension on longitudinal, patient-centric Electronic Health Records (EHRs) across multiple datasets, models and metrics, covering extractive, multiple-choice, and open-ended generative QA under different context formulation and Retrieval Augmented Generation (RAG) settings. We show that medically fine-tuned models offer no measurable benefit over base counterparts, smaller models exhibit stronger memorization, and longitudinal supplementary notes aid generative QA. Reasoning models gain on complex multi-note QA under LLM Correctness but not stricter reference-aligned metrics, while hybrid RAG with reranking outperforms Full-Context (FC) prompting. A question-category analysis and multi-faceted metric evaluation further reveal where RAG and FC excel and struggle, with RAG falling behind in longitudinal reasoning, thus uncovering systematic failures pointing to the need for reasoning-appropriate metrics.

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v3,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v3 · 2026-11-13 06:58 · 最新观测版本

Investigating LLM Capabilities on Long Context Comprehension for Clinical Question Answering

Despite the impressive reported performance of Large Language Models (LLMs) on medical Question-Answering (QA), evaluation still overrelies on multiple-choice and extractive settings primarily assessing textbook knowledge, with limited work on open-ended QA over patient records. We present a comprehensive study investigating LLM long-context (LC) comprehension on longitudinal, patient-centric Electronic Health Records (EHRs) across multiple datasets, models and metrics, covering extractive, multiple-choice, and open-ended generative QA under different context formulation and Retrieval Augmented Generation (RAG) settings. We show that medically fine-tuned models offer no measurable benefit over base counterparts, smaller models exhibit stronger memorization, and longitudinal supplementary notes aid generative QA. Reasoning models gain on complex multi-note QA under LLM Correctness but not stricter reference-aligned metrics, while hybrid RAG with reranking outperforms Full-Context (FC) prompting. A question-category analysis and multi-faceted metric evaluation further reveal where RAG and FC excel and struggle, with RAG falling behind in longitudinal reasoning, thus uncovering systematic failures pointing to the need for reasoning-appropriate metrics.

查看此版本解读 →查看此版本原文 ↗