NatSGLD：一个包含语音、手势、逻辑和演示的机器人学习数据集，用于自然人机交互
NatSGLD: A Dataset with Speech, Gesture, Logic, and Demonstration for Robot Learning in Natural Human-Robot Interaction
AI总结：
针对现有HRI数据集局限于简单任务的问题，NatSGLD采用WoZ方法收集语音、手势、演示轨迹及LTL公式，为多模态指令跟随和计划识别等研究提供基础资源。
摘要：
近年来，多模态人机交互（HRI）数据集的最新进展强调语音和手势的整合，使机器人能够吸收显性知识和隐性理解。然而，现有数据集主要关注诸如物体指向和推动等基础任务，限制了其在复杂领域的适用性。这些数据集优先考虑更简单的人类命令数据，但较少强调训练机器人正确理解任务并做出适当响应。为解决这些不足，我们提出了NatSGLD数据集，该数据集采用“巫师之oz”（WoZ）方法收集，参与者与他们认为自主的机器人进行交互。NatSGLD记录了人类的多模态命令（语音和手势），每条命令都配有一个演示轨迹和一个线性时序逻辑（LTL）公式，该公式提供了对命令任务的真实解释。该数据集作为人机交互与机器学习交叉研究的基础资源。通过提供多模态输入和详细标注，NatSGLD支持多模态指令跟随、计划识别以及从演示中的人类可建议强化学习等领域的探索。我们在MIT许可证下于https://www.snehesh.com/natsgld/发布该数据集和代码，以支持未来的HRI研究。
英文摘要：
Recent advances in multimodal Human-Robot Interaction (HRI) datasets emphasize the integration of speech and gestures, allowing robots to absorb explicit knowledge and tacit understanding. However, existing datasets primarily focus on elementary tasks like object pointing and pushing, limiting their applicability to complex domains. They prioritize simpler human command data but place less emphasis on training robots to correctly interpret tasks and respond appropriately. To address these gaps, we present the NatSGLD dataset, which was collected using a Wizard of Oz (WoZ) method, where participants interacted with a robot they believed to be autonomous. NatSGLD records humans' multimodal commands (speech and gestures), each paired with a demonstration trajectory and a Linear Temporal Logic (LTL) formula that provides a ground-truth interpretation of the commanded tasks. This dataset serves as a foundational resource for research at the intersection of HRI and machine learning. By providing multimodal inputs and detailed annotations, NatSGLD enables exploration in areas such as multimodal instruction following, plan recognition, and human-advisable reinforcement learning from demonstrations. We release the dataset and code under the MIT License at https://www.snehesh.com/natsgld/ to support future HRI research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
