arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.03388cs.ROcs.CLcs.LG
收藏

用于机器人物体消歧的LLMs

LLMs for Robotic Object Disambiguation

Massachusetts Institute of Technology(麻省理工学院);Smart Systems Institute, National University of Singapore(新加坡国立大学智能系统研究所)

Connie Jiang, Yiqing Xu, David Hsu

AI总结:

本研究探索将LLM集成至桌面机器人消歧任务,针对零样本提示的不足开发少样本提示系统,使模型能有效利用并推断特征以在复杂物体簇中精准决策并检索目标物体。

摘要:

预训练大型语言模型(LLMs)在各种语言处理任务中的优势是显而易见的。但是,能否进一步利用语言模型的知识来有效消歧物体并应对机器人领域中的决策挑战?我们的研究揭示了LLM解决复杂决策挑战的资质,这些挑战通常此前由部分可观察马尔可夫决策过程(POMDPs)建模。我们研究的一个关键重点是LLMs的物体消歧能力。我们详细介绍了将LLM集成到桌面环境消歧任务中的过程,这是一个决策问题,机器人的任务是从任意大且复杂的物体簇中辨别并检索用户期望的物体。尽管使用零样本提示工程进行了多次查询尝试(详情见附录),LLM仍难以询问场景描述中未明确提供的特征。作为回应,我们开发了一个少样本提示工程系统,以提高LLM提出消歧查询的能力。其结果是一个模型,既能在可用时使用给定特征,又能在必要时推断新的相关特征,从而成功生成并沿着精确的决策树导航到正确的物体——即使面临相同的选项也是如此。

英文摘要:

The advantages of pre-trained large language models (LLMs) are apparent in a variety of language processing tasks. But can a language model's knowledge be further harnessed to effectively disambiguate objects and navigate decision-making challenges within the realm of robotics? Our study reveals the LLM's aptitude for solving complex decision making challenges that are often previously modeled by Partially Observable Markov Decision Processes (POMDPs). A pivotal focus of our research is the object disambiguation capability of LLMs. We detail the integration of an LLM into a tabletop environment disambiguation task, a decision making problem where the robot's task is to discern and retrieve a user's desired object from an arbitrarily large and complex cluster of objects. Despite multiple query attempts with zero-shot prompt engineering (details can be found in the Appendix), the LLM struggled to inquire about features not explicitly provided in the scene description. In response, we have developed a few-shot prompt engineering system to improve the LLM's ability to pose disambiguating queries. The result is a model capable of both using given features when they are available and inferring new relevant features when necessary, to successfully generate and navigate down a precise decision tree to the correct object--even when faced with identical options.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑