arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.20116cs.HCcs.AI
收藏

Estuary:一个用于构建多模态低延迟实时社交互动智能体的框架

Estuary: A Framework For Building Multimodal Low-Latency Real-Time Socially Interactive Agents

University of Southern California(南加州大学)

Spencer Lin, Basem Rizk, Miru Jun, Andy Artze, Caitlin Sullivan, Sharon Mozgai, Scott Fisher

AI总结:

针对实时社交互动智能体缺乏通用框架的问题,本文提出Estuary,以模块化、可离线运行的多模态架构支持低延迟实时SIA开发并提升可复现性。

摘要:

生成式人工智能(AI)技术能力的提升及其普及,使其得以应用于社交互动智能体(Socially Interactive Agents,SIAs)领域。尽管人们对用于实时SIA研究的现代AI驱动组件的兴趣不断增长,但由于缺乏标准化、通用的SIA框架,仍存在显著阻碍。针对这一缺失,我们开发了Estuary:一个多模态(文本、音频,视频支持即将推出)框架,用于促进低延迟、实时SIA的开发。Estuary旨在减少不同研究之间的重复工作,并提供一个可完全在云端之外运行的灵活平台,以最大化可配置性、可控性、研究可复现性以及智能体响应速度。我们能够实现这一点,是通过构建一个稳健的多模态框架,将当前和未来的组件无缝整合到模块化、可互操作的架构中。

英文摘要:

The rise in capability and ubiquity of generative artificial intelligence (AI) technologies has enabled its application to the field of Socially Interactive Agents (SIAs). Despite rising interest in modern AI-powered components used for real-time SIA research, substantial friction remains due to the absence of a standardized and universal SIA framework. To target this absence, we developed Estuary: a multimodal (text, audio, and soon video) framework which facilitates the development of low-latency, real-time SIAs. Estuary seeks to reduce repeat work between studies and to provide a flexible platform that can be run entirely off-cloud to maximize configurability, controllability, reproducibility of studies, and speed of agent response times. We are able to do this by constructing a robust multimodal framework which incorporates current and future components seamlessly into a modular and interoperable architecture.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑