In this paper we introduce a knowledge engine, which learns and shares knowledge representations, for robots to carry out a variety of tasks. Building such an engine brings with it the challenge of dealing with multiple data modalities including symbols, natural language, haptic senses, robot trajectories, visual features and many others. The \textit{knowledge} stored in the engine comes from multiple sources including physical interactions that robots have while performing tasks (perception, planning and control), knowledge bases from the Internet and learned representations from several robotics research groups.
We discuss various technical aspects and associated challenges such as modeling the correctness of knowledge, inferring latent information and formulating different robotic tasks as queries to the knowledge engine. We describe the system architecture and how it supports different mechanisms for users and robots to interact with the engine. Finally, we demonstrate its use in three important research areas: grounding natural language, perception, and planning, which are the key building blocks for many robotic tasks. This knowledge engine is a collaborative effort and we call it RoboBrain.
A Comparison of Prompt Engineering Techniques for Task Planning and Execution in Service Robotics
University of Bonn(波恩大学);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习与人工智能研究所);Center for Robotics(机器人中心);Computer Science Institute VI(计算机科学第六研究所);Autonomous Intelligent Systems group(自主智能系统组)
FluxVLA Engine: A One-Stop VLA Engineering Platform for Embodied Intelligence
LimX Dynamics;Nankai University(南开大学);The University of Hong Kong(香港大学);University of Electronic Science and Technology of China(电子科技大学);Beijing Institute of Technology(北京理工大学);Xidian University(西安电子科技大学);Zhejiang University(浙江大学);Xi’an Jiaotong University(西安交通大学)