arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.00481cs.NIcs.AIcs.HCcs.MMcs.PF
收藏

用LLM进行视频通话:对六个主流应用的测量活动

Make a Video Call with LLM: A Measurement Campaign over Six Mainstream Apps

Hong Kong University of Science and Technology(香港科技大学)

Jiayang Xu, Xiangjie Huang, Zijie Li, Antariksh Verma, Zili Meng

AI总结:

本文通过自定义测试平台和在线平台,从质量、延迟、内部机制和系统开销四个维度对六个主流AI视频聊天应用进行基准测试,发现AI视频通话的网络延迟影响小于人类视频通话,AI代理能力对用户体验影响最大。

摘要:

2025年,大型语言模型(LLM)服务推出了一项新功能——AI视频聊天,允许用户通过实时视频通信(RTC)与AI代理互动,就像与真人聊天一样。尽管其重要性,但尚无系统性研究描述现有AI视频聊天系统的性能。为填补这一空白,本文提出了一个涵盖四个维度的综合基准:质量、延迟、内部机制和系统开销。使用自定义测试平台,我们进一步用该基准评估了六个主流AI视频聊天机器人。我们还构建了一个用于用户研究的在线平台。测量结果得出了有趣的发现,可能对未来优化有益。例如,AI视频通话的网络延迟不如人类视频通话重要。AI代理的能力对用户体验影响最大。我们的基准测试结果也为未来AI视频聊天机器人的优化提出了几个研究问题。可用性:在线评估平台、开源数据集和测试平台见https://callarena.net/。

英文摘要:

In 2025, Large Language Model (LLM) services have launched a new feature -- AI video chat -- allowing users to interact with AI agents via real-time video communication (RTC), just like chatting with real people. Despite its significance, no systematic study has characterized the performance of existing AI video chat systems. To address this gap, this paper proposes a comprehensive benchmark across four dimensions: quality, latency, internal mechanisms, and system overhead. Using custom testbeds, we further evaluate six mainstream AI video chatbots with this benchmark. We also build an online platform for user study. The measurement leads to interesting findings that could be beneficial to the future optimizations. For example, the network latency of AI video chat matters not as much as human video chat. The capabilities of AI agents matters most in the user experience. Our benchmarking results also open up several research questions for future optimizations of AI video chatbots. Availability: https://callarena.net/ for the online evaluation platform and our open-sourced dataset and testbed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑