基于动作片段无监督学习的机器人行为字幕生成
Caption Generation of Robot Behaviors based on Unsupervised Learning of Action Segments
摘要：
将机器人动作序列与其自然语言字幕连接起来是提高人类辅助机器人可解释性的一项重要任务。在本文中，我们提出了一个系统，用于生成自然语言字幕，描述人类辅助机器人的行为。该系统描述了机器人的行动，通过使用机器人的意见，历史从致动器系统和摄像机，机器人的行动和自然语言字幕之间的端到端的桥接。有两个原因使得将现有的序列到序列模型应用于这种映射具有挑战性：1）很难为任何类型的机器人及其环境准备大规模数据集，以及2）从机器人动作观察获得的样本数量与生成的字幕单词序列之间存在差距。我们引入了基于K-means聚类的无监督分割，将典型的机器人观察模式统一到一类中。这种方法使网络能够从少量数据中学习关系。此外，我们利用了一种基于字节对编码（BPE）的组块方法，以填补机器人动作观察的样本数量和标题中的单词之间的差距。我们还将注意力机制应用于分割任务。实验结果表明，该模型基于无监督学习可以产生更好的描述比其他方法。我们还表明，注意力机制并没有在我们的低资源设置很好地工作。
英文摘要：
Bridging robot action sequences and their natural language captions is an important task to increase explainability of human assisting robots in their recently evolving field. In this paper, we propose a system for generating natural language captions that describe behaviors of human assisting robots. The system describes robot actions by using robot observations; histories from actuator systems and cameras, toward end-to-end bridging between robot actions and natural language captions. Two reasons make it challenging to apply existing sequence-to-sequence models to this mapping: 1) it is hard to prepare a large-scale dataset for any kind of robots and their environment, and 2) there is a gap between the number of samples obtained from robot action observations and generated word sequences of captions. We introduced unsupervised segmentation based on K-means clustering to unify typical robot observation patterns into a class. This method makes it possible for the network to learn the relationship from a small amount of data. Moreover, we utilized a chunking method based on byte-pair encoding (BPE) to fill in the gap between the number of samples of robot action observations and words in a caption. We also applied an attention mechanism to the segmentation task. Experimental results show that the proposed model based on unsupervised learning can generate better descriptions than other methods. We also show that the attention mechanism did not work well in our low-resource setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
