Humanoid robots have significant gaps in their sensing and perception, making it hard to perform motion planning in dense environments. To address this, we introduce ARMOR, a novel egocentric perception system that integrates both hardware and software, specifically incorporating wearable-like depth sensors for humanoid robots. Our distributed perception approach enhances the robot's spatial awareness, and facilitates more agile motion planning. We also train a transformer-based imitation learning (IL) policy in simulation to perform dynamic collision avoidance, by leveraging around 86 hours worth of human realistic motions from the AMASS dataset. We show that our ARMOR perception is superior against a setup with multiple dense head-mounted, and externally mounted depth cameras, with a 63.7% reduction in collisions, and 78.7% improvement on success rate. We also compare our IL policy against a sampling-based motion planning expert cuRobo, showing 31.6% less collisions, 16.9% higher success rate, and 26x reduction in computational latency. Lastly, we deploy our ARMOR perception on our real-world GR1 humanoid from Fourier Intelligence. We are going to update the link to the source code, HW description, and 3D CAD files in the arXiv version of this text.
Reflex: Visually Grounded Reactive Humanoid Control
University of Washington（华盛顿大学）；Stanford University（斯坦福大学）；University of Southern California（南加州大学）；National University of Singapore（新加坡国立大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Allen Institute for AI（艾伦人工智能研究所）
ViLoMan: Learning Visual-Proprioceptive Whole-Body Loco-Manipulation Skills for Humanoid Robots
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS（中国科学院计算技术研究所人工智能安全国家重点实验室）；University of Chinese Academy of Sciences (CAS)（中国科学院大学）；Beijing Academy of Artificial Intelligence (BAAI)（北京智源人工智能研究院）
EgoPriMo: Egocentric Motion Generation for Interactive Humanoid Control
Tianjin University（天津大学）；Zhongguancun Academy（中关村学院）；Beihang University（北京航空航天大学）；Zhongguancun Institute of Artificial Intelligence（中关村人工智能研究院）；DeepCybo
ZeroWBC: Learning Natural Whole-Body Humanoid Interaction from Human Egocentric Data
University of Science and Technology of China（中国科学技术大学）；Shanghai AI Laboratory（上海人工智能实验室）；Northwestern Polytechnical University（西北工业大学）；Tsinghua University（清华大学）；Shanghai Jiao Tong University（上海交通大学）；TeleAI, China Telecom（TeleAI，中国电信）
CoHuB: A Simulation Benchmark for Multi-Humanoid Collaboration
Yonsei University（延世大学）；Gwangju Institute of Science and Technology（光州科学技术院）；University of California, Berkeley（加州大学伯克利分校）；Seoul National University（首尔大学）
Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining
Alibaba Group（阿里巴巴集团）；Sichuan University（四川大学）；Shanghai Innovation Institute（上海创新研究院）；Beihang University（北京航空航天大学）；Hong Kong University of Science and Technology（香港科技大学）
Gait-Adaptive Perceptive Humanoid Locomotion with Real-Time Under-Base Terrain Reconstruction
Department of Electronic Engineering and Information Science (EEIS), University of Science and Technology of China（电子工程与信息科学系，中国科学技术大学）；LimX Dynamics（LimX动力学）；Hong Kong University of Science and Technology（香港科技大学）；School of Mechanics Engineering, Harbin Institute of Technology (HIT)（机械工程学院，哈尔滨工业大学）；Department of Computing, The Hong Kong Polytechnic University（计算学院，香港理工大学）；Zhejiang University-University of Illinois Urbana-Champaign Institute (ZJUI)（浙江大学-伊利诺伊大学厄巴纳-香槟分校联合研究所）
Human-as-Humanoid: Enabling Zero-Shot Humanoid Learning from Ego-Exo Human Videos with Human-Aligned Embodiments
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；DeepCybo；ZGCA；ZGCI；Harbin Institute of Technology（哈尔滨工业大学）；Huazhong University of Science and Technology（华中科技大学）；Beihang University（北京航空航天大学）
We aim to build complex humanoid agents that integrate perception, motor control, and memory. In this work, we partly factor this problem into low-level motor control from proprioception and high-level coordination of the low-level skills informed by vision. We develop an architecture capable of surprisingly flexible, task-directed motor control of a relatively high-DoF humanoid body by combining pre-training of low-level motor controllers with a high-level, task-focused controller that switches among low-level sub-policies. The resulting system is able to control a physically-simulated humano…
EgoHumanoid-V2: Human-to-Humanoid Transfer of Coordinated Whole-Body Skills for Loco-Manipulation
OpenDriveLab at The University of Hong Kong（香港大学OpenDriveLab）；Alibaba Group（阿里巴巴集团）；Shanghai Innovation Institute（上海创新研究院）；Fudan University（复旦大学）；Beihang University（北京航空航天大学）；Sichuan University（四川大学）；Archon Robotics（Archon机器人公司）
RoboMirror: Understand Before You Imitate for Video to Humanoid Locomotion
Nanyang Technological University（南洋理工大学）；The Hong Kong University of Science and Technology（香港科技大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Xi’an Jiaotong University（西安交通大学）；Chinese Academy of Sciences（中国科学院）；Shanghai Jiao Tong University（上海交通大学）；University of Sydney（悉尼大学）；Peking University（北京大学）
PhysBrain: Human Egocentric Data as a Bridge from Vision Language Models to Physical Intelligence
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Zhongguancun Academy（中关村学院）；Zhongguancun Institute of Artificial Intelligence（中关村人工智能研究院）；Harbin Institute of Technology（哈尔滨工业大学）；Huazhong University of Science and Technology（华中科技大学）
OminiAdapt: Learning Cross-Task Invariance for Robust and Environment-Aware Robotic Manipulation
University of Science and Technology of China（中国科学技术大学）；Beijing Institute of Technology（北京理工大学）；Institute of Automation Chinese Academy of Sciences（中国科学院自动化研究所）
Institute for Systems and Robotics (ISR/IST)（系统与机器人研究所（ISR/IST））；Instituto Superior Tecnico（高等技术学院）；University Lisboa（里斯本大学）
The ability to distinguish between the self and the background is of paramount importance for robotic tasks. The particular case of hands, as the end effectors of a robotic system that more often enter into contact with other elements of the environment, must be perceived and tracked with precision to execute the intended tasks with dexterity and without colliding with obstacles. They are fundamental for several applications, from Human-Robot Interaction tasks to object manipulation. Modern humanoid robots are characterized by high number of degrees of freedom which makes their forward kinemat…
AnyWorld: Factorized Egocentric World Models for Cross-Embodiment Generalization
Nanyang Technological University（南洋理工大学）；Institute of Advanced Intelligence and Computing, A*STAR（新加坡科技研究局高级智能与计算研究院）；XPENG Robotics（小鹏机器人）；Zhejiang University（浙江大学）；The Chinese University of Hong Kong（香港中文大学）
