← 研踪 PaperTrailarXiv:2606.21453 · v2
Robustness of Text-Only Adapted Speech LLMs for Spoken Semantic Parsing
首次提交 2026-06-19 22:11
版本更新 2026-11-18 21:14
雷达观测 2026-11-19 15:04
摘要
Large language models (LLMs) are increasingly used to generate structured outputs from natural language instructions, including semantic frames and function calls. When the input is speech, this task becomes spoken semantic parsing, which can be performed using either Speech LLMs or ASR-LLM cascades. Although both approaches can be adapted with text-only task supervision, their relative robustness to acoustic degradation remains unclear. We compare these approaches under matched text-form supervision using 3B and 7B models across three datasets, evaluated on clean speech and babble noise at five SNR levels. Results show that Speech LLMs generally achieve better performance in command understanding, intent prediction, and function calling, particularly under severe noise, while improvements in fine-grained entity prediction are less consistent. An additional evaluation shows that Speech LLMs achieve lower average inference time per sample than ASR-LLM cascades at both model scales.
作者备注:review in progress
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-18 21:14 · 最新观测版本
Robustness of Text-Only Adapted Speech LLMs for Spoken Semantic Parsing
Large language models (LLMs) are increasingly used to generate structured outputs from natural language instructions, including semantic frames and function calls. When the input is speech, this task becomes spoken semantic parsing, which can be performed using either Speech LLMs or ASR-LLM cascades. Although both approaches can be adapted with text-only task supervision, their relative robustness to acoustic degradation remains unclear. We compare these approaches under matched text-form supervision using 3B and 7B models across three datasets, evaluated on clean speech and babble noise at five SNR levels. Results show that Speech LLMs generally achieve better performance in command understanding, intent prediction, and function calling, particularly under severe noise, while improvements in fine-grained entity prediction are less consistent. An additional evaluation shows that Speech LLMs achieve lower average inference time per sample than ASR-LLM cascades at both model scales.
查看此版本解读 →查看此版本原文 ↗