This paper describes a framework called MaestROB. It is designed to make the robots perform complex tasks with high precision by simple high-level instructions given by natural language or demonstration. To realize this, it handles a hierarchical structure by using the knowledge stored in the forms of ontology and rules for bridging among different levels of instructions. Accordingly, the framework has multiple layers of processing components; perception and actuation control at the low level, symbolic planner and Watson APIs for cognitive capabilities and semantic understanding, and orchestration of these components by a new open source robot middleware called Project Intu at its core. We show how this framework can be used in a complex scenario where multiple actors (human, a communication robot, and an industrial robot) collaborate to perform a common industrial task. Human teaches an assembly task to Pepper (a humanoid robot from SoftBank Robotics) using natural language conversation and demonstration. Our framework helps Pepper perceive the human demonstration and generate a sequence of actions for UR5 (collaborative robot arm from Universal Robots), which ultimately performs the assembly (e.g. insertion) task.
Human-Robot Interaction in GenAI Architectures via the Agent-Client Protocol
Machine Perception and Intelligent Robotics Group (MAPIR-UMA), Malaga Institute for Mechatronics Engineering and Cyber-Physical Systems (IMECH.UMA), University of Malaga(机器感知与智能机器人组(MAPIR-UMA),马拉加机电一体化工程与网络物理系统研究所(IMECH.UMA),马拉加大学)
Speculative Policy Orchestration: A Latency-Resilient Framework for Cloud-Robotic Manipulation
Department of Computing Science, Umeå University(乌梅大学计算科学系);School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology(皇家理工学院电气工程与计算机科学学院)
FrankenBot: Brain-Morphic Modular Orchestration for Robotic Manipulation with Vision-Language Models
School of Future Technology, South China University of Technology(未来技术学院,华南理工大学);School of Software Engineering, South China University of Technology(软件工程学院,华南理工大学);Shien-Ming Wu School of Intelligent Engineering, South China University of Technology(智能工程学院,华南理工大学)
Transforming Monolithic Foundation Models into Embodied Multi-Agent Architectures for Human-Robot Collaboration
Department of Computer Science and Technology, Tsinghua University(计算机科学与技术系,清华大学);School of Artificial Intelligence, Beijing University of Posts and Telecommunications(人工智能学院,北京邮电大学)
The More the Merrier: Running Multiple Neuromorphic Components On-Chip for Robotic Control
fortiss – Research Institute of the Free State of Bavaria(巴伐利亚州自由州fortiss研究所);Ludwig Maximillian University of Munich(慕尼黑大学);Intel Neuromorphic Computing Lab(英特尔神经形态计算实验室);Technical University of Munich(慕尼黑工业大学)
SwiftBot: A Decentralized Platform for LLM-Powered Federated Robotic Task Execution
California State University, Long Beach, USA(加州州立大学长滩分校);Case Western Reserve University, USA(凯斯西储大学);University of Colorado Denver, USA(科罗拉多大学丹佛分校);Montana State University, USA(蒙塔那州立大学);University of Houston Clear Lake, USA(休斯顿大学clear lake分校)
When Should a Robot Think? Resource-Aware Reasoning via Reinforcement Learning for Embodied Robotic Decision-Making
Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所);Northeastern University(东北大学);Harvard University(哈佛大学);Cornell University(康奈尔大学);MIT(麻省理工学院);Fujitsu Research of America(美国富士通研究);Tsinghua University(清华大学);Peking University(北京大学);University of Georgia(佐治亚大学);Florida International University(佛罗里达国际大学);EmbodyX Inc(EmbodyX公司);Cisco Systems(思科系统)
UniManip: Open-World Robotic Manipulation via Agentic Operational Graphs
School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore(南洋理工大学电子与电气工程学院);Department of Automation, Tsinghua University, China(清华大学自动化系)
LLM-Empowered Embodied Agent for Memory-Augmented Task Planning in Household Robotics
Automation and Control Institute, Faculty of Electrical Engineering, TU Wien(自动化与控制研究所,电气工程学院,维也纳技术大学);AIT Austrian Institute of Technology GmbH, Center for Vision, Automation and Control(奥地利技术研究所(AIT)有限公司,视觉、自动化与控制中心)
Towards Agentic Robots: Model, Data, Environment, and Harness
MMLab, The Chinese University of Hong Kong(香港中文大学MMLab);Fudan University(复旦大学);NovaXBot;Tsinghua University(清华大学);Zhejiang University(浙江大学);Sun Yat-sen University(中山大学)
Toward Evidence-Driven Human-Agent-Robot Teaming for Earth-Independent Anomaly Triage
NASA Ames Research Center(美国国家航空航天局艾姆斯研究中心);SETI Institute(搜寻地外文明研究所);The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Michigan(密歇根大学);Repertory Robotics