arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.03701cs.RO
收藏

ExTraCT —— 使用特征的文本描述从语言输入进行可解释的轨迹修正

ExTraCT -- Explainable Trajectory Corrections from language inputs using Textual description of features

Nanyang Technological University (NTU)(南洋理工大学);Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局);National Healthcare Group (NHG)(新加坡国立健保集团)

J-Anne Yow, Neha Priyadarshini Garg, Manoj Ramanathan, Wei Tech Ang

AI总结:

本文提出ExTraCT框架,结合LLM和轨迹形变函数,通过自然语言实现可解释的轨迹修正,解决端到端方法泛化性差的问题,并在用户研究中优于基线。

摘要:

自然语言提供了一种直观且富有表现力的方式,将人类意图传达给机器人。先前的工作采用端到端方法从语言修正中学习轨迹形变。然而,此类方法无法泛化到新的初始轨迹或物体配置。本研究提出了ExTraCT,这是一个使用自然语言进行轨迹修正的模块化框架,它结合了用于自然语言理解的大型语言模型(LLM)和轨迹形变函数。给定一个场景,ExTraCT基于模板在线生成轨迹修改特征(场景特定和场景无关)以及场景中物体的相应自然语言文本描述。我们使用LLM将用户话语与特征的文本描述进行语义匹配。基于匹配的特征,将轨迹修改函数应用于初始轨迹,从而能够泛化到未见过的轨迹和物体配置。通过在仿真和物理机械臂中进行的用户研究,我们证明使用我们方法形变的轨迹更准确,并且在约80%的情况下更受青睐,优于基线方法。我们还展示了我们的系统在操作任务和辅助喂食任务中的多功能性。

英文摘要:

Natural language provides an intuitive and expressive way of conveying human intent to robots. Prior works employed end-to-end methods for learning trajectory deformations from language corrections. However, such methods do not generalize to new initial trajectories or object configurations. This work presents ExTraCT, a modular framework for trajectory corrections using natural language that combines Large Language Models (LLMs) for natural language understanding and trajectory deformation functions. Given a scene, ExTraCT generates the trajectory modification features (scene-specific and scene-independent) and their corresponding natural language textual descriptions for the objects in the scene online based on a template. We use LLMs for semantic matching of user utterances to the textual descriptions of features. Based on the feature matched, a trajectory modification function is applied to the initial trajectory, allowing generalization to unseen trajectories and object configurations. Through user studies conducted both in simulation and with a physical robot arm, we demonstrate that trajectories deformed using our method were more accurate and were preferred in about 80\% of cases, outperforming the baseline. We also showcase the versatility of our system in a manipulation task and an assistive feeding task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑