We aim to build complex humanoid agents that integrate perception, motor control, and memory. In this work, we partly factor this problem into low-level motor control from proprioception and high-level coordination of the low-level skills informed by vision. We develop an architecture capable of surprisingly flexible, task-directed motor control of a relatively high-DoF humanoid body by combining pre-training of low-level motor controllers with a high-level, task-focused controller that switches among low-level sub-policies. The resulting system is able to control a physically-simulated humanoid body to solve tasks that require coupling visual perception from an unstabilized egocentric RGB camera during locomotion in the environment. For a supplementary video link, see https://youtu.be/7GISvfbykLE .
MetaWorld-X: Hierarchical World Modeling via VLM-Orchestrated Experts for Humanoid Loco-Manipulation
University of Hamburg（汉堡大学）；School of Information Science and Technology, Beijing University of Technology（信息科学与技术学院，北京理工大学）；School of Information Science and Engineering, Fudan University（信息科学与工程学院，复旦大学）；University of Alberta（阿尔伯塔大学）
LeVERB: Humanoid Whole-Body Control with Latent Vision-Language Instruction
University of California Berkeley（加州大学伯克利分校）；Norwegian University of Science and Technology（挪威科学与技术大学）；Simon Fraser University（西蒙弗雷泽大学）；Carnegie Mellon University（卡内基梅隆大学）
Hierarchical generative modelling for autonomous robots
Intel Labs（英特尔实验室）；University College London（伦敦大学学院）；Wellcome Centre for Human Neuroimaging（惠康神经影像中心）；Queen Square Institute of Neurology（皇后广场神经学研究所）；UCL Robotics Institute（伦敦大学学院机器人研究所）
