This paper addresses the challenge of humanoid robot teleoperation in a natural indoor environment via a Brain-Computer Interface (BCI). We leverage deep Convolutional Neural Network (CNN) based image and signal understanding to facilitate both real-time bject detection and dry-Electroencephalography (EEG) based human cortical brain bio-signals decoding. We employ recent advances in dry-EEG technology to stream and collect the cortical waveforms from subjects while they fixate on variable Steady State Visual Evoked Potential (SSVEP) stimuli generated directly from the environment the robot is navigating. To these ends, we propose the use of novel variable BCI stimuli by utilising the real-time video streamed via the on-board robot camera as visual input for SSVEP, where the CNN detected natural scene objects are altered and flickered with differing frequencies (10Hz, 12Hz and 15Hz). These stimuli are not akin to traditional stimuli - as both the dimensions of the flicker regions and their on-screen position changes depending on the scene objects detected. On-screen object selection via such a dry-EEG enabled SSVEP methodology, facilitates the on-line decoding of human cortical brain signals, via a specialised secondary CNN, directly into teleoperation robot commands (approach object, move in a specific direction: right, left or back). This SSVEP decoding model is trained via a priori offline experimental data in which very similar visual input is present for all subjects. The resulting classification demonstrates high performance with mean accuracy of 85% for the real-time robot navigation experiment across multiple test subjects.
Reinforcement Learning with Data Bootstrapping for Dynamic Subgoal Pursuit in Humanoid Robot Navigation
Mechanical and Aerospace Engineering, The Ohio State University(机械与航空航天工程,俄亥俄州立大学);Electrical and Computer Engineering, The Ohio State University(电气与计算机工程,俄亥俄州立大学);Computer Science and Engineering, The Ohio State University(计算机科学与工程,俄亥俄州立大学)
Learning to Evolve: Multi-modal Interactive Fields for Robust Humanoid Navigation in Dynamic Environments
State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School(一般人工智能国家重点实验室,北京大学深圳研究生院);Oxford Robotics Institute, University of Oxford(牛津大学机器人研究所);Institute for Machine Learning, Department of Computer Science, ETH Zurich(苏黎世联邦理工学院计算机科学系机器学习研究所)
FlashNav: Training Deployable Robot Navigation Policies in Seconds
Eastern Institute of Technology, Ningbo(宁波东方理工大学);The Hong Kong Polytechnic University(香港理工大学);National University of Singapore(新加坡国立大学);University of Science and Technology of China(中国科学技术大学);Shanghai Jiao Tong University(上海交通大学)
FSUNav: A Cerebrum-Cerebellum Architecture for Fast, Safe, and Universal Zero-Shot Goal-Oriented Navigation
School of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机科学学院);College of Information Science and Technology, Eastern Institute of Technology(东方理工学院信息科学与技术学院)
School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所人工智能安全国家重点实验室);National Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所模式识别国家重点实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);XYZ Embodied AI(XYZ具身人工智能)
RAVEN: Reinforcement-Adaptive Visibility-Graph Planning for Robust Humanoid Navigation with Collision-Free MPC
Robotics and Mechanisms Laboratory (RoMeLa), Department of Mechanical and Aerospace Engineering, University of California, Los Angeles(机器人与机构实验室(RoMeLa),机械与航天工程系,加利福尼亚大学洛杉矶分校)
AgniNav: Configuration-Driven Cross-Embodiment Local Planning for Robot Navigation
Eastern Institute of Technology, Ningbo, China(东方理工(宁波));University of Nottingham, Nottingham, UK(诺丁汉大学);University of Science and Technology of China, Hefei, China(中国科学技术大学)
Gallant: Voxel Grid-based Humanoid Locomotion and Local-navigation across 3D Constrained Terrains
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);University of Science and Technology of China(中国科学技术大学);University of Tokyo(东京大学);Shanghai Jiaotong University(上海交通大学)
TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model
University of California, Berkeley(加州大学伯克利分校);Peking University(北京大学);Tsinghua University(清华大学);The University of Hong Kong(香港大学);Princeton University(普林斯顿大学)
Humanoid Path Planning over Rough Terrain using Traversability Assessment
Institute of Human and Machine Cognition(人类与机器认知研究所);University of West Florida(西佛罗里达大学);Pusan National University(釜山国立大学);Boardwalk Robotics, Inc.(Boardwalk机器人公司)
我们提出了一个规划框架,旨在人形导航具有挑战性的地形。该框架旨在使用2.5D高度图规划可遍历,平滑和无碰撞的路径。规划器由两个阶段组成。第一阶段由A* 规划器组成,该规划器使用地形特征对可通行性进行推理。提出了一种新的成本函数,它编码的双足步态直接到图形结构,使自然的路径,是强大的小差距的可通行性。第二阶段是一个优化框架,在进一步提高可遍历性的同时平滑路径。该规划器在模拟中的各种地形上进行了测试,并与脚步规划器和平衡控制器相结合,以创建一个集成的导航框架,该框架在DRC Boston Dynamics Atlas机器人上进行了演示。
End-to-End Humanoid Robot Safe and Comfortable Locomotion Policy
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Human-Robot Interfaces and Interaction Lab., Istituto Italiano di Tecnologia, Italy(人机交互实验室,意大利理工学院)
HumanoidPano: Hybrid Spherical Panoramic-LiDAR Cross-Modal Perception for Humanoid Robots
Beijing Innovation Center of Humanoid Robotics(北京人形机器人创新中心);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Hong Kong University of Science and Technology(香港科技大学);Beijing Institute of Technology(北京理工大学);ETH Zurich(苏黎世联邦理工学院);Nanyang Technological University(南洋理工大学);X-Humanoid(人形智能科技公司)
Efficient Image-Goal Navigation with Representative Latent World Model
College of Intelligence Science and Technology, the National Key Laboratory of Equipment State Sensing and Smart Support, National University of Defense Technology(智能科学与技术学院、装备状态感知与智能支持国家重点实验室、国防科技大学)