发表机构
Harvard University（哈佛大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
AVERT结合跨轮次一致性与音频条件验证器，通过三类操作解决口语对话状态跟踪的三类错误，在SpokenWOZ上JGA达40.13，性能接近10亿参数端到端系统。
AI 中文摘要
口语对话状态跟踪任务需从语音中恢复槽值对，自动语音识别（ASR）错误多集中在实体值且会跨轮次持续存在，因此该任务兼具生成与编辑属性。强大的单轮文本编辑器可修正大部分此类错误，但仅基于转录文本操作时，仍会留下三类可恢复错误：跨轮次预测不一致的值、缺失的槽位、音频不支持的值。本文提出AVERT，该机制通过将跨轮次一致性与训练后的音频条件验证器相结合，对每个候选值进行打分，并通过投票、添加、交换三类操作分别解决这三类错误，每类操作仅适用于其错误常见的槽位子集。在SpokenWOZ数据集上，基础语音大语言模型（speech-LLM）的联合目标准确率（JGA）为33.04，文本编辑器为38.34，AVERT达到40.13，且无需对前两者进行重新训练。该性能与消耗完整口语历史的10亿参数端到端系统（JGA为39.32）相当，不过AVERT使用两个10亿参数解码器而非一个。音频验证器带来了具有统计显著性的性能提升，且将每类操作限制在选定槽位子集至关重要：若移除该限制，无约束投票会覆盖正确的分类值，导致性能低于文本编辑器。
英文摘要
Spoken dialogue state tracking recovers slot-value pairs from speech, where ASR errors concentrate in entity values and persist across turns, making it both a generation and an editing problem. A strong per-turn text editor corrects much of this but, operating on the transcript alone, leaves three recoverable errors: a value predicted inconsistently across turns, an omitted slot, and a value the audio does not support. We present AVERT, which scores each candidate value by combining cross-turn agreement with a trained audio-conditioned verifier and resolves the three error types with three operators, vote, add, and swap, each restricted to the slots where its error is common. On SpokenWOZ, a base speech-LLM reaches 33.04 JGA, a text editor 38.34, and AVERT 40.13, without retraining either. This is in the range of a 1B end-to-end system that consumes the full spoken history (39.32), though AVERT uses two 1B decoders rather than one. The audio verifier contributes a statistically significant gain, and restricting each operator to a selected slot subset matters: removing it lets unrestricted voting overwrite correct categorical values and fall below the editor.
