Jarvis：面向多方对话的主动式语音智能体
Jarvis: A Proactive Speech Agent for Multi-Party Conversations
浏览论文内容
中文总结 AI 辅助
Jarvis是一个实时主动式语音智能体，基于共享文档参与多方对话，在小组遗漏或错误陈述事实时主动干预，并通过CHI-180-proactive数据集和实时研究验证了其有效性与高沉默率。
中文摘要 AI 辅助
语音智能体通常是反应式和双人式的：它们只在被呼叫时发言，并且一次只与一个人交流。我们探讨的是，语音智能体要如何才能转而参与多人对话，并且只在能够提供帮助时才主动发言。我们推出了Jarvis，一个实时主动式语音智能体，能够以语音形式参与多人人类对话。Jarvis基于事先共享的文档，跟踪讨论进程，并在小组遗漏或错误陈述某个事实且未在几轮对话内自行纠正时进行干预。我们做出了三项贡献：一是基于认知断裂的问题设定，使主动干预变得可衡量，并具体化为CHI-180-proactive，这是一个包含已知空白、错误和自我纠正的合成多方数据集；二是主动式主干架构，利用小型开放权重模型配合确定性检查，并将每个论断都基于源句子；三是交互技术，用于在实时语音中抢占发言权并在屏幕上显示引用的证据。在CHI-180-proactive上，Jarvis对其处理的大多数事件判断正确，并在小组自行解决问题时97%的时间保持沉默。一项包含23名参与者的实时研究证实了这些趋势在实时干预中的有效性。
英文摘要
Speech agents are reactive and dyadic: they speak when spoken to, and to one person at a time. We ask what it takes for a speech agent to instead take part in a conversation among several people and speak up only when it can help. We introduce Jarvis, a real-time proactive speech agent that audibly participates in multi-party human conversations. Grounded in a document shared beforehand, Jarvis follows the discussion and intervenes when the group misses or misstates a fact and does not correct itself within a few turns. We make three contributions: a problem setting based on epistemic breakdowns that makes proactive intervention measurable, realized as CHI-180-proactive, a synthetic multi-party dataset seeded with known gaps, errors, and self-corrections; a proactive backbone that harnesses a small, open-weight model with deterministic checks and grounds every claim in a source sentence; and interaction techniques for taking the floor in live speech and showing the cited evidence on screen. On CHI-180-proactive, Jarvis is correct on most events it addresses and stays silent 97% of the time when the group resolves an issue itself. A live study with 23 participants confirms these trends with real-time interventions.
发表机构
- University of Washington（华盛顿大学）
- The University of Tokyo（东京大学）
机构由 AI 辅助整理，请以论文原文为准。
