In order to detect and correct physical exercises, a Grow-When-Required Network (GWR) with recurrent connections, episodic memory and a novel subnode mechanism is developed in order to learn spatiotemporal relationships of body movements and poses. Once an exercise is performed, the information of pose and movement per frame is stored in the GWR. For every frame, the current pose and motion pair is compared against a predicted output of the GWR, allowing for feedback not only on the pose but also on the velocity of the motion. In a practical scenario, a physical exercise is performed by an expert like a physiotherapist and then used as a reference for a humanoid robot like Pepper to give feedback on a patient's execution of the same exercise. This approach, however, comes with two challenges. First, the distance from the humanoid robot and the position of the user in the camera's view of the humanoid robot have to be considered by the GWR as well, requiring a robustness against the user's positioning in the field of view of the humanoid robot. Second, since both the pose and motion are dependent on the body measurements of the original performer, the expert's exercise cannot be easily used as a reference. This paper tackles the first challenge by designing an architecture that allows for tolerances in translation and rotations regarding the center of the field of view. For the second challenge, we allow the GWR to grow online on incremental data. For evaluation, we created a novel exercise dataset with virtual avatars called the Virtual-Squat dataset. Overall, we claim that our novel architecture based on the GWR can use a learned exercise reference for different body variations through continual online learning, while preventing catastrophic forgetting, enabling for an engaging long-term human-robot interaction with a humanoid robot.
Humanoid Robots as Agents of Human Consciousness Expansion
Hanson Robotics(汉森机器人公司);OpenCog Foundation(OpenCog基金会);Institute of Noetic Sciences(心智科学研究所);Mossbridge Institute, LLC(莫斯布里奇研究所有限责任公司);Hong Kong Polytechnic University(香港理工大学)
Never-Ending Behavior-Cloning Agent for Robotic Manipulation
State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences(机器人国家重点实验室,沈阳自动化研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学);School of Automation Science and Engineering, South China University of Technology(自动化科学与工程学院,华南理工大学);Mohamed bin Zayed University of Artificial Intelligence(马斯达尔大学人工智能学院)
Extreme-RGMT: Continual Learning of Highly Dynamic Skills for Robust Generalist Humanoid Control
School of Automation, Beijing Institute of Technology(北京理工大学自动化学院);Humanoid Robotics (Shanghai) Co., Ltd.(人形机器人(上海)有限公司);School of Mechanical Engineering, Shandong University(山东大学机械工程学院)
Towards Human-AI-Robot Collaboration and AI-Agent based Digital Twins for Parkinson's Disease Management: Review and Outlook
Computer, Electrical and Mathematical Sciences and Engineering Division (CEMSE), King Abdullah University of Science and Technology(计算机、电气与数学科学和工程系,国王 Abdullah 科学与技术大学);College of Engineering & Advanced Computing and the Translational Biomedical Engineering Research lab, Alfaisal University(工程与高级计算学院及转化生物医学工程研究实验室,阿尔法西尔大学);Electrical Engineering Department and the Center for Communication Systems and Sensing, King Fahd University of Petroleum and Minerals (KFUPM)(电气工程系及通信系统与感知中心,国王法赫德石油与矿物大学)
Simulation and Retargeting of Complex Multi-Character Interactions
Georgia Institute of Technology(佐治亚理工学院);Apple(苹果公司);Reality Labs Research, Meta(Meta现实实验室研究部门);Carnegie Mellon University(卡内基梅隆大学);Seoul National University(首尔国立大学)