arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.01804cs.CVcs.AI
收藏

非语言实时人机交互在受限机器人环境中

Non-verbal Real-time Human-AI Interaction in Constrained Robotic Environments

National University of Science;Princeton University, Princeton NJ 08544, USA, , WWW home page: http://users/ iekeland/web/welcome.html Universit\' e de Paris-Sud, Laboratoire d'Analyse Num\' e rique, B\ a timent 425, F-91405 Orsay Cedex, France

Dragos Costea, Alina Marcu, Cristina Lazar, Marius Leordeanu

AI总结:

本文提出首个实时生成非语言人机交互的框架,通过对比合成与真实数据发现时间一致性对实际性能的影响,揭示了人类与AI运动间的统计差异。

摘要:

我们研究了在全身体态下非语言通信中人工智能生成数据与人类生成数据统计保真度的持续争论。具体来说,我们问当代生成模型是否超越了表面模仿,参与到无声但富有表现力的身体语言对话中。我们通过引入第一个框架来解决这个问题,该框架能够从2D身体关键点实时生成自然的非语言人机交互。我们的实验使用了四个轻量级架构,能够在NVIDIA Orin Nano上以最高100帧每秒的速度运行,有效地关闭了感知-动作回路,这对于自然的人机交互至关重要。我们训练了437个人类视频片段,并证明了在合成序列上预训练可以显著减少运动误差,而不会牺牲速度。然而,仍然存在可测量的现实差距。当最佳模型在提取自最新文本到视频系统(如SORA和VEO)的关键点上进行评估时,我们发现其在SORA生成的片段上表现下降,但在VEO上下降得远少,这表明时间一致性,而非图像保真度,驱动了现实中的性能。我们的结果表明,人类和人工智能运动之间仍然存在统计上可区分的差异。

英文摘要:

We study the ongoing debate regarding the statistical fidelity of AI-generated data compared to human-generated data in the context of non-verbal communication using full body motion. Concretely, we ask if contemporary generative models move beyond surface mimicry to participate in the silent, but expressive dialogue of body language. We tackle this question by introducing the first framework that generates a natural non-verbal interaction between Human and AI in real-time from 2D body keypoints. Our experiments utilize four lightweight architectures which run at up to 100 FPS on an NVIDIA Orin Nano, effectively closing the perception-action loop needed for natural Human-AI interaction. We trained on 437 human video clips and demonstrated that pretraining on synthetically-generated sequences reduces motion errors significantly, without sacrificing speed. Yet, a measurable reality gap persists. When the best model is evaluated on keypoints extracted from cutting-edge text-to-video systems, such as SORA and VEO, we observe that performance drops on SORA-generated clips. However, it degrades far less on VEO, suggesting that temporal coherence, not image fidelity, drives real-world performance. Our results demonstrate that statistically distinguishable differences persist between Human and AI motion.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑