DaDu-E: Rethinking the Role of Large Language Model in Robotic Computing Pipeline
Institute of Computing Technology, Chinese Academy of Sciences (ICT, CAS)(中国科学院计算技术研究所);University of Chinese Academy of Sciences (UCAS)(中国科学院大学);Beijing Institute of Technology (BIT)(北京理工大学);Institute of Automation, Chinese Academy of Science (CASIA)(中国科学院自动化研究所);Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)(深圳市人工智能与机器人研究院)
Wenhao Sun, Sai Hou, Zixuan Wang, Bo Yu, Shaoshan Liu, Xu Yang, Shuai Liang, Yiming Gan, Yinhe Han
Performing complex tasks in open environments remains challenging for robots, even when using large language models (LLMs) as the core planner. Many LLM-based planners are inefficient due to their large number of parameters and prone to inaccuracies because they operate in open-loop systems. We think the reason is that only applying LLMs as planners is insufficient. In this work, we propose DaDu-E, a robust closed-loop planning framework for embodied AI robots. Specifically, DaDu-E is equipped with a relatively lightweight LLM, a set of encapsulated robot skill instructions, a robust feedback system, and memory augmentation. Together, these components enable DaDu-E to (i) actively perceive and adapt to dynamic environments, (ii) optimize computational costs while maintaining high performance, and (iii) recover from execution failures using its memory and feedback mechanisms. Extensive experiments on real-world and simulated tasks show that DaDu-E achieves task success rates comparable to embodied AI robots with larger models as planners like COME-Robot, while reducing computational requirements by $6.6 \times$. Users are encouraged to explore our system at: \url{https://rlc-lab.github.io/dadu-e/}.
DaDu-Corki: Algorithm-Architecture Co-Design for Embodied AI-powered Robotic Manipulation
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳人工智能与机器人研究院);Meituan(美团);Tianjin University(天津大学)
Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verification
School of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机科学学院);Computer Science, King Abdullah University of Science and Technology(阿卜杜拉国王科技大学计算机科学学院)
Stanford University(斯坦福大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);NVIDIA Research(英伟达研究院);Princeton University(普林斯顿大学);Carnegie Mellon University(卡内基梅隆大学)
Rethinking Temporal Object Detection from Robotic Perspectives
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Beijing Innovation Center for Engineering Science and Advanced Technology, Peking University(北京大学工程科学与先进技术北京创新中心);Beihang University(北京航空航天大学)
Rethinking Bimanual Robotic Manipulation: Learning with Decoupled Interaction Framework
School of Computer Science and Engineering, Sun Yat-sen University, China(中山大学计算机科学与工程学院);Peng Cheng Laboratory, China(鹏城实验室);Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China(教育部人工智能与先进计算重点实验室);Guangdong Province Key Laboratory of Information Security Technology, China(广东省信息安全技术重点实验室);Imperial-X Initiative and Department of Bioengineering, Imperial College London, U.K.(帝国理工学院生物工程系)
Point Cloud Matters: Rethinking the Impact of Different Observation Spaces on Robot Learning
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);Northwestern Polytechnical University(西北工业大学);Zhejiang University(浙江大学)