Non-verbal Real-time Human-AI Interaction in Constrained Robotic Environments
National University of Science;Princeton University, Princeton NJ 08544, USA, , WWW home page: http://users/ iekeland/web/welcome.html Universit\' e de Paris-Sud, Laboratoire d'Analyse Num\' e rique, B\ a timent 425, F-91405 Orsay Cedex, France
Dragos Costea, Alina Marcu, Cristina Lazar, Marius Leordeanu
我们研究了在全身体态下非语言通信中人工智能生成数据与人类生成数据统计保真度的持续争论。具体来说,我们问当代生成模型是否超越了表面模仿,参与到无声但富有表现力的身体语言对话中。我们通过引入第一个框架来解决这个问题,该框架能够从2D身体关键点实时生成自然的非语言人机交互。我们的实验使用了四个轻量级架构,能够在NVIDIA Orin Nano上以最高100帧每秒的速度运行,有效地关闭了感知-动作回路,这对于自然的人机交互至关重要。我们训练了437个人类视频片段,并证明了在合成序列上预训练可以显著减少运动误差,而不会牺牲速度。然而,仍然存在可测量的现实差距。当最佳模型在提取自最新文本到视频系统(如SORA和VEO)的关键点上进行评估时,我们发现其在SORA生成的片段上表现下降,但在VEO上下降得远少,这表明时间一致性,而非图像保真度,驱动了现实中的性能。我们的结果表明,人类和人工智能运动之间仍然存在统计上可区分的差异。
英文摘要:
We study the ongoing debate regarding the statistical fidelity of AI-generated data compared to human-generated data in the context of non-verbal communication using full body motion. Concretely, we ask if contemporary generative models move beyond surface mimicry to participate in the silent, but expressive dialogue of body language. We tackle this question by introducing the first framework that generates a natural non-verbal interaction between Human and AI in real-time from 2D body keypoints. Our experiments utilize four lightweight architectures which run at up to 100 FPS on an NVIDIA Orin Nano, effectively closing the perception-action loop needed for natural Human-AI interaction. We trained on 437 human video clips and demonstrated that pretraining on synthetically-generated sequences reduces motion errors significantly, without sacrificing speed. Yet, a measurable reality gap persists. When the best model is evaluated on keypoints extracted from cutting-edge text-to-video systems, such as SORA and VEO, we observe that performance drops on SORA-generated clips. However, it degrades far less on VEO, suggesting that temporal coherence, not image fidelity, drives real-world performance. Our results demonstrate that statistically distinguishable differences persist between Human and AI motion.
Transforming Surgical Interventions with Embodied Intelligence for Ultrasound Robotics
Centre for Artificial Intelligence and Robotics (CAIR), HKISI-CAS(香港中文大学(深圳)人工智能与机器人中心(HKISI-CAS));Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
GMatch: A Lightweight, Geometry-Constrained Keypoint Matcher for Zero-Shot 6DoF Pose Estimation in Robotic Grasp Tasks
The State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China(多模态人工智能系统国家重点实验室,自动化研究所,中国科学院,北京)
CT-SAFR: Safe and Interpretable Chain-of-Thought Reasoning for Autonomous Robots: A Multi-Layered Verification Framework for Trustworthy AI-Driven Robotic Decision Making
SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects (Early Version)
Tuojing Intelligence(拓景智能);Tsinghua University(清华大学);King’s College London(伦敦国王学院);Southeast University(东南大学);Stevens Institute of Technology(史蒂文斯理工学院);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));University of Manchester(曼彻斯特大学);Simple AI(简单人工智能公司);Imperial College London(伦敦帝国学院);Carnegie Mellon University(卡内基梅隆大学);Zhejiang University(浙江大学);Beihang University(北京航空航天大学);The University of Hong Kong(香港大学)
When a Robot is More Capable than a Human: Learning from Constrained Demonstrators
Thomas Lord Department of Computer Science, University of Southern California(汤姆·劳德计算机科学系,南加州大学);Meta AI;Department of Computer Science & Engineering, University of California San Diego(计算机科学与工程系,加州大学圣地亚哥分校)
LiPS: Lightweight Panoptic Segmentation for Resource-Constrained Robotics
Université Paris-Saclay, CEA LIST(巴黎-萨克雷大学,CEA LIST);U2IS, ENSTA, Institut Polytechnique de Paris(U2IS、ENSTA、巴黎理工学院);University of Bonn, Center for Robotics(波恩大学,机器人中心)