arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.02772cs.ROcs.AIcs.HC
收藏

用于自然人机通信的跨模态力与语言嵌入

Cross-modality Force and Language Embeddings for Natural Human-Robot Communication

Electrical Engineering and Computer Science Massachusetts Institute of Technology(电气工程与计算机科学麻省理工学院);Department of Physical Medicine, Harvard Medical School Rehabilitation, Spaulding Rehabilitation Hospital(物理医学系哈佛医学院康复科,斯波尔丁康复医院);Mechanical Engineering Massachusetts Institute of Technology(机械工程麻省理工学院)

Ravi Tejwani, Karl Velazquez, John Payne, Paolo Bonato, Harry Asada

AI总结:

本文提出一种将力曲线与单词统一嵌入到潜在空间的跨模态框架,以协同协调语言与触觉通信,实现人机之间类似人类搬物般有效的自然交互。

摘要:

本文提出了一种力曲线与单词的跨模态嵌入方法,用于协同协调语言与触觉通信。当两人共同搬运大型重物时,他们通过关于预期运动和施加在物体上的物理力的语言交流进行协调。这种语言与物理线索的自然整合实现了有效的协调。类似地,人机交互也可以通过整合语言与触觉通信模态来实现这种水平的协调。本文提出了一个以统一方式嵌入单词和力曲线的框架,使得这两种通信模态能够以有效且协同的方式进行整合与协调。本文将证明,尽管语言与物理力曲线被认为截然不同,但两者可以嵌入统一的潜在空间中,并且两者之间的邻近度可以被量化。在该潜在空间中,力曲线和单词可以a)相互补充,b)整合各自的效果,c)以可互换的方式相互替代。首先,探讨了跨模态嵌入的需求,并介绍了基本架构和关键基础构建技术。随后讨论了数据收集方法和实现挑战,最后给出了实验结果与讨论。

英文摘要:

A method for cross-modality embedding of force profile and words is presented for synergistic coordination of verbal and haptic communication. When two people carry a large, heavy object together, they coordinate through verbal communication about the intended movements and physical forces applied to the object. This natural integration of verbal and physical cues enables effective coordination. Similarly, human-robot interaction could achieve this level of coordination by integrating verbal and haptic communication modalities. This paper presents a framework for embedding words and force profiles in a unified manner, so that the two communication modalities can be integrated and coordinated in a way that is effective and synergistic. Here, it will be shown that, although language and physical force profiles are deemed completely different, the two can be embedded in a unified latent space and proximity between the two can be quantified. In this latent space, a force profile and words can a) supplement each other, b) integrate the individual effects, and c) substitute in an exchangeable manner. First, the need for cross-modality embedding is addressed, and the basic architecture and key building block technologies are presented. Methods for data collection and implementation challenges will be addressed, followed by experimental results and discussions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑