基于LLM的端到端口语对话状态跟踪的联合语音与文本训练
Joint Speech and Text Training for LLM-Based End-to-End Spoken Dialogue State Tracking
AI总结：
提出联合训练口语DST数据和跨域文本DST数据的方法，解决口语数据稀缺和跨域泛化问题，无需目标域口语训练数据即可获得良好性能。
摘要：
端到端口语对话状态跟踪（DST）因需要处理语音输入和数据稀缺而变得困难。最近的工作提出结合语音基础编码器和大型语言模型来缓解这一困难。尽管这种方法已被证明能产生强大的口语DST模型，在现实多轮DST中达到最先进性能，但它难以跨域泛化，并且需要每个目标域的口语DST训练数据。然而，为每个目标域收集此类数据既昂贵又困难。注意到文本DST数据更容易从各种领域获得，在这项工作中，我们提出联合训练可用的口语DST数据和其他领域的书面文本数据，以实现跨域泛化。我们进行的实验表明，我们提出的方法在无需目标域口语训练数据的情况下，能够获得良好的跨域DST性能。
英文摘要：
End-to-end spoken dialogue state tracking (DST) is made difficult by the tandem of having to handle speech input and data scarcity. Combining speech foundation encoders and large language models has been proposed in recent work as to alleviate some of this difficulty. Although this approach has been shown to result in strong spoken DST models, achieving state-of-the-art performance in realistic multi-turn DST, it struggles to generalize across domains and requires annotated spoken DST training data for each domain of interest. However, collecting such data for every target domain is both costly and difficult. Noting that textual DST data is more easily obtained for various domains, in this work, we propose jointly training on available spoken DST data and written textual data from other domains as a way to achieve cross-domain generalization. We conduct experiments which show the efficacy of our proposed method for getting good cross-domain DST performance without relying on spoken training data from the target domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
