面向对话AI模型的多模态上下文对话故障检测
Multimodal Contextual Dialogue Breakdown Detection for Conversational AI Models
AI总结：
针对对话AI系统的实时对话故障检测需求，尤其是医疗等行业场景的高精度高灵活要求，本文提出融合音频输入与下游NLP文本推理的MultConDB多模态模型，其F1值达69.27，显著优于现有最优模型。
摘要：
实时检测对话故障对对话AI系统至关重要，因为它能支持系统采取纠正措施以顺利完成任务。在口语对话系统中，这类故障可能由多种意外情况引发，包括高强度背景噪声导致STT（语音转文本）转写错误，或是用户的对话流程不符合预期。尤其是在医疗等行业场景中，系统需要具备高精度和高灵活性，以根据对话历史与对话状态调整应对方式。这使得准确检测对话故障既更具挑战性，也更为关键。我们发现，要实现准确的故障检测，需要实时处理音频输入，同时结合下游NLP模型对转写文本的推理结果。本文提出了Multimodal Contextual Dialogue Breakdown（多模态上下文对话故障，MultConDB）模型，该模型的F1值达到69.27，性能显著优于其他已知的最优模型。
英文摘要：
Detecting dialogue breakdown in real time is critical for conversational AI systems, because it enables taking corrective action to successfully complete a task. In spoken dialog systems, this breakdown can be caused by a variety of unexpected situations including high levels of background noise, causing STT mistranscriptions, or unexpected user flows. In particular, industry settings like healthcare, require high precision and high flexibility to navigate differently based on the conversation history and dialogue states. This makes it both more challenging and more critical to accurately detect dialog breakdown. To accurately detect breakdown, we found it requires processing audio inputs along with downstream NLP model inferences on transcribed text in real time. In this paper, we introduce a Multimodal Contextual Dialogue Breakdown (MultConDB) model. This model significantly outperforms other known best models by achieving an F1 of 69.27.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
