Humans and animals excel in combining information from multiple sensory modalities, controlling their complex bodies, adapting to growth, failures, or using tools. These capabilities are also highly desirable in robots. They are displayed by machines to some extent. Yet, the artificial creatures are lagging behind. The key foundation is an internal representation of the body that the agent - human, animal, or robot - has developed. The mechanisms of operation of body models in the brain are largely unknown and even less is known about how they are constructed from experience after birth. In collaboration with developmental psychologists, we conducted targeted experiments to understand how infants acquire first "sensorimotor body knowledge". These experiments inform our work in which we construct embodied computational models on humanoid robots that address the mechanisms behind learning, adaptation, and operation of multimodal body representations. At the same time, we assess which of the features of the "body in the brain" should be transferred to robots to give rise to more adaptive and resilient, self-calibrating machines. We extend traditional robot kinematic calibration focusing on self-contained approaches where no external metrology is needed: self-contact and self-observation. Problem formulation allowing to combine several ways of closing the kinematic chain simultaneously is presented, along with a calibration toolbox and experimental validation on several robot platforms. Finally, next to models of the body itself, we study peripersonal space - the space immediately surrounding the body. Again, embodied computational models are developed and subsequently, the possibility of turning these biologically inspired representations into safe human-robot collaboration is studied.
ZeroWBC: Learning Natural Whole-Body Humanoid Interaction from Human Egocentric Data
University of Science and Technology of China(中国科学技术大学);Shanghai AI Laboratory(上海人工智能实验室);Northwestern Polytechnical University(西北工业大学);Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学);TeleAI, China Telecom(TeleAI,中国电信)
State Estimation for Human Motion and Humanoid Locomotion
Istituto Italiano di Tecnologia(意大利理工学院);IRI CSIC-UPC(西班牙国家研究委员会-加泰罗尼亚理工大学机器人研究所);AIST(日本产业技术综合研究所);LAAS-CNRS(法国国家科学研究中心系统分析与架构实验室);DIBRIS-UNIGE(热那亚大学计算机、生物工程、机器人与系统工程系)
VividFace: Real-Time and Realistic Facial Expression Shadowing for Humanoid Robots
School of Computing, Macquarie University(麦考瑞大学计算机学院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算机与数据科学学院);Anuradha and Vikas Sinha Department of Data Science, University of North Texas(北卡罗来纳州立大学数据科学系)
Istituto Italiano di Tecnologia(意大利理工学院);Florida Institute for Human and Machine Cognition(佛罗里达人机认知研究所);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);National Institute of Advanced Industrial Science and Technology (AIST)(日本产业技术综合研究所);Inria(法国国家信息与自动化研究所);Université de Lorraine(洛林大学);CNRS(法国国家科学研究中心);Tokyo University of Science(东京理科大学)
MuJoCo MPC for Humanoid Control: Evaluation on HumanoidBench
Technical University of Darmstadt(达姆施塔特工业大学);German Research Center for AI (DFKI)(德国人工智能研究中心);Centre for Cognitive Science(认知科学中心);Hessian Center for Artificial Intelligence (Hessian.AI)(黑森州人工智能中心)
Humanoid Factors: Design Principles for AI Humanoids in Human Worlds
School of Computing and Augmented Intelligence, Arizona State University, AZ, USA(计算与增强智能学院,亚利桑那州立大学,亚利桑那州,美国);School of Manufacturing Systems and Networks, Arizona State University, AZ, USA(制造系统与网络学院,亚利桑那州立大学,亚利桑那州,美国);Human Systems Engineering, Arizona State University, AZ, USA(人机系统工程,亚利桑那州立大学,亚利桑那州,美国)
TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model
University of California, Berkeley(加州大学伯克利分校);Peking University(北京大学);Tsinghua University(清华大学);The University of Hong Kong(香港大学);Princeton University(普林斯顿大学)
KungfuBot2: Learning Versatile Motion Skills for Humanoid Whole-Body Control
Institute of Artificial Intelligence (TeleAI), China Telecom(人工智能研究院(TeleAI),中国电信);Shanghai Jiao Tong University(上海交通大学);East China University of Science and Technology(东华大学)
Human-as-Humanoid: Enabling Zero-Shot Humanoid Learning from Ego-Exo Human Videos with Human-Aligned Embodiments
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));DeepCybo;ZGCA;ZGCI;Harbin Institute of Technology(哈尔滨工业大学);Huazhong University of Science and Technology(华中科技大学);Beihang University(北京航空航天大学)