ConceptBot: Enhancing Robot's Autonomy through Task Decomposition with Large Language Models and Knowledge Graph
Department of Innovative Technologies (DTI), Dalle Molle Institute for Artificial Intelligence (IDSIA), University of Applied Sciences(创新技术部门(DTI)、达摩迪人工智能研究所(IDSIA)、应用科学与艺术大学);Department of Mechanical Engineering, Politecnico di Milano(机械工程部门、米兰理工学院)
Alessandro Leanza, Angelo Moroncelli, Giuseppe Vizzari, Francesco Braghin, Loris Roveda, Blerina Spahiu
ConceptBot是一种模块化机器人规划框架,它结合大语言模型(Large Language Models, LLM)与知识图谱,在自然语言指令存在歧义、且需正确分析环境中物体的情况下生成可行且具备风险感知的规划——这些挑战通常源于常识推理能力的缺失。为此,ConceptBot集成了三个模块:(1)物体属性提取(Object Property Extraction, OPE)模块,利用ConceptNet的语义概念丰富场景理解;(2)用户请求处理(User Request Processing, URP)模块,对指令进行消歧并结构化;(3)规划器,生成具备上下文感知能力的可行抓取-放置策略。在与Google SayCan的对比评估中,ConceptBot在显式任务上取得100%成功率,在隐式任务上保持87%准确率(SayCan为31%),在风险感知任务上达到76%(SayCan为15%),且在材料分类(70% vs 20%)、毒性检测(86% vs 36%)等特定应用场景中均优于SayCan。在SafeAgentBench基准上,ConceptBot的整体得分为80%(次优基线为46%)。这些结果在仿真与实验室实验中均得到验证,表明ConceptBot无需领域特定训练即可泛化,能显著提升非结构化环境中机器人策略的可靠性。项目网站:https://sites.google.com/view/conceptbot
英文摘要:
ConceptBot is a modular robotic planning framework that combines Large Language Models and Knowledge Graphs to generate feasible and risk-aware plans despite ambiguities in natural language instructions and correctly analyzing the objects present in the environment - challenges that typically arise from a lack of commonsense reasoning. To do that, ConceptBot integrates (i) an Object Property Extraction (OPE) module that enriches scene understanding with semantic concepts from ConceptNet, (ii) a User Request Processing (URP) module that disambiguates and structures instructions, and (iii) a Planner that generates context-aware, feasible pick-and-place policies. In comparative evaluations against Google SayCan, ConceptBot achieved 100% success on explicit tasks, maintained 87% accuracy on implicit tasks (versus 31% for SayCan), reached 76% on risk-aware tasks (versus 15%), and outperformed SayCan in application-specific scenarios, including material classification (70% vs. 20%) and toxicity detection (86% vs. 36%). On SafeAgentBench, ConceptBot achieved an overall score of 80% (versus 46% for the next-best baseline). These results, validated in both simulation and laboratory experiments, demonstrate ConceptBot's ability to generalize without domain-specific training and to significantly improve the reliability of robotic policies in unstructured environments. Website: https://sites.google.com/view/conceptbot
Trust-Preserved Human-Robot Shared Autonomy enabled by Bayesian Relational Event Modeling
School of Electrical and Computer Engineering, Georgia Institute of Technology(电子与计算机工程学院,佐治亚理工学院);Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology(电子与计算机工程系,香港科技大学)
Trust-Aware Embodied Bayesian Persuasion for Mixed-Autonomy
Electrical & Computer Engineering, University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校电气与计算机工程系);Industrial & Enterprise Systems Engineering, University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校工业与企业系统工程系)
DTRT: Enhancing Human Intent Estimation and Role Allocation for Physical Human-Robot Collaboration
CAS Engineering Laboratory for Intelligent Industrial Vision, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所CAS工程视觉实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Large language model-based task planning for service robots: A review
School of Electrical Engineering, Yanshan University(燕山大学电气工程学院);Engineering Research Center of Intelligent Control System and Intelligent Equipment, Ministry of Education, Yanshan University(燕山大学智能控制与设备工程研究中心);Hebei Key Laboratory of Intelligent Rehabilitation and Neuromodulation, Yanshan University(河北省智能康复与神经调制重点实验室);School of Control Science and Engineering, Shandong University(山东大学控制科学与工程学院);National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University(国家机器人视觉感知与控制技术工程研究中心);Department of Automation, Shanghai Jiao Tong University(上海交通大学自动化系);Advanced Robotics and Intelligent Systems Laboratory, School of Engineering, University of Guelph(圭尔夫大学工程学院先进机器人与智能系统实验室)
School of Intelligent Systems Science and Engineering(智能系统科学与工程学院);Jinan University(济南大学);School of Artificial Intelligence(人工智能学院);Jilin University(吉林大学);Shanghai Innovation Institute(上海创新研究院);Lehigh University(莱文斯顿大学);University of Illinois Chicago(伊利诺伊大学芝加哥分校)