跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2609.17355 · v2

Evaluating Ambient Clinical Scribes in India: The Need for Multilingual Real-World Clinical Conversation Data

首次提交 2026-09-15 23:52
版本更新 2026-11-16 16:12
雷达观测 2026-11-17 17:11
arXiv ↗PDF ↗

摘要

Ambient clinical scribes (ACS) are being rapidly deployed at scale across Global South healthcare settings to reduce clinicians' documentation time, especially in overburdened environments such as India. These ACS are primarily developed or distilled from models built and validated on Global North speech, languages and consultation styles. Indian clinical encounters are brief, triadic, multilingual, code-mixed with low-resource languages, and conducted in highly resource-constrained, noisy settings -- increasing the likelihood of ASR and note-generation errors manyfold. We argue for a standardized evaluation infrastructure to assess whether these systems are safe, reliable, and well-suited to the Indian healthcare setting. We substantiate our claims through a mixed-methods study -- a systematic survey of publicly available patient-clinician conversational datasets, a quantitative comparison of some simulated datasets against conversational and cultural markers drawn from the Indian clinical-communication literature, and semi-structured interviews with four organizations building and deploying ACS in India. Our survey shows that there are no large-scale, publicly available, real-world benchmarks for ACS in India, with existing datasets being overwhelmingly synthetic or simulated. We note that amongst the available simulated datasets that we compare, the Global North datasets diverge significantly from the expected conversational and cultural structures of their Indian encounters. Finally, our interviews reveal that deploying organizations have each built proprietary, incomparable evaluation pipelines, creating a fragmented ecosystem with no independent and reliable basis for procurement. We call for the development of a publicly shared, real-world, multilingual benchmark for ACS evaluation and outline the properties and policies such a benchmark would require.

作者备注:Accepted to Findings Track of ML4H 2026

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-16 16:12 · 最新观测版本

Evaluating Ambient Clinical Scribes in India: The Need for Multilingual Real-World Clinical Conversation Data

Ambient clinical scribes (ACS) are being rapidly deployed at scale across Global South healthcare settings to reduce clinicians' documentation time, especially in overburdened environments such as India. These ACS are primarily developed or distilled from models built and validated on Global North speech, languages and consultation styles. Indian clinical encounters are brief, triadic, multilingual, code-mixed with low-resource languages, and conducted in highly resource-constrained, noisy settings -- increasing the likelihood of ASR and note-generation errors manyfold. We argue for a standardized evaluation infrastructure to assess whether these systems are safe, reliable, and well-suited to the Indian healthcare setting. We substantiate our claims through a mixed-methods study -- a systematic survey of publicly available patient-clinician conversational datasets, a quantitative comparison of some simulated datasets against conversational and cultural markers drawn from the Indian clinical-communication literature, and semi-structured interviews with four organizations building and deploying ACS in India. Our survey shows that there are no large-scale, publicly available, real-world benchmarks for ACS in India, with existing datasets being overwhelmingly synthetic or simulated. We note that amongst the available simulated datasets that we compare, the Global North datasets diverge significantly from the expected conversational and cultural structures of their Indian encounters. Finally, our interviews reveal that deploying organizations have each built proprietary, incomparable evaluation pipelines, creating a fragmented ecosystem with no independent and reliable basis for procurement. We call for the development of a publicly shared, real-world, multilingual benchmark for ACS evaluation and outline the properties and policies such a benchmark would require.

查看此版本解读 →查看此版本原文 ↗