TopoNav: Topological Navigation for Efficient Exploration in Sparse Reward Environments
University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；Amazon Inc.（亚马逊公司）；DEVCOM Army Research Lab（DEVCOM陆军研究实验室）；Stormfish Scientific Corporation（斯托姆菲什科学公司）
Autonomous robots exploring unknown environments face a significant challenge: navigating effectively without prior maps and with limited external feedback. This challenge intensifies in sparse reward environments, where traditional exploration techniques often fail. In this paper, we present TopoNav, a novel topological navigation framework that integrates active mapping, hierarchical reinforcement learning, and intrinsic motivation to enable efficient goal-oriented exploration and navigation in sparse-reward settings. TopoNav dynamically constructs a topological map of the environment, capturing key locations and pathways. A two-level hierarchical policy architecture, comprising a high-level graph traversal policy and low-level motion control policies, enables effective navigation and obstacle avoidance while maintaining focus on the overall goal. Additionally, TopoNav incorporates intrinsic motivation to guide exploration toward relevant regions and frontier nodes in the topological map, addressing the challenges of sparse extrinsic rewards. We evaluate TopoNav both in the simulated and real-world off-road environments using a Clearpath Jackal robot, across three challenging navigation scenarios: goal-reaching, feature-based navigation, and navigation in complex terrains. We observe an increase in exploration coverage by 7- 20%, in success rates by 9-19%, and reductions in navigation times by 15-36% across various scenarios, compared to state-of-the-art methods
TopoNav: Topological Graphs as a Key Enabler for Advanced Object Navigation
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；Beijing Innovation Center of Humanoid Robotics Co., Ltd.（北京人形机器人创新中心有限公司）；Shenzhen International Graduate School, Tsinghua University（深圳国际研究生学院，清华大学）
Autonomous navigation of catheters and guidewires in mechanical thrombectomy using inverse reinforcement learning
School of Biomedical Engineering & Imaging Sciences（生物医学工程与影像科学学院）；Kings College London（伦敦国王学院）；AIBE；Friedrich-Alexander University Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）
Confidence-Controlled Exploration: Efficient Sparse-Reward Policy Learning for Robot Navigation
U.S. Army Research Laboratory（美国陆军研究实验室）；JP Morgan AI Research（摩根大通人工智能研究院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Central Florida（中佛罗里达大学）
Vision-Language Procedural Reasoning for Context-Aware Reward Modeling of Robotic Endovascular Guidewire Navigation
Department of Control Science and Engineering, College of Electronic and Information Engineering, and Shanghai Institute of Intelligent Science and Technology, Tongji University（同济大学电子与信息工程学院控制科学与工程系，以及上海智能科学与技术研究院）；Department of Electronic Engineering, Faculty of Engineering, The Chinese University of Hong Kong（香港中文大学工程学院电子工程系）；Shanghai Operation Robot Co., Ltd.（上海微创医疗机器人（集团）股份有限公司）
Reinforcement learning reward function in unmanned aerial vehicle control tasks
Bauman Moscow State Technical University（鲍曼莫斯科国立技术大学）；Semenov Federal Research Center for Chemical Physics of the Russian Academy of Sciences（俄罗斯科学院谢苗诺夫化学物理联邦研究中心）
SPLC: Social Preference Learning for Crowd Robot Navigation
Wuhan University of Science and Technology（武汉科技大学）；Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System（湖北省智能信息处理与实时工业系统重点实验室）
Robotics Research Center, IIIT Hyderabad（印度信息技术与管理研究所海得拉巴分校机器人研究中心）；TCS Research, Tata Consultancy Services（塔塔咨询服务公司TCS研究院）；University of Birmingham（伯明翰大学）
Capsule Network Performance with Autonomous Navigation
Purdue University（普渡大学）
胶囊网络（CapsNets）已经被提出作为卷积神经网络（CNN）的替代方案。本文展示了CapsNets如何比CNN更有能力进行自主智能体对现实场景的探索。在现实世界的导航，奖励外部代理可能是罕见的。反过来，强化学习算法可能难以形成有意义的策略函数。本文的方法Capsules Exploration Module（Caps-EM）将CapsNets架构与Advantage Actor Critic算法配对。用于导航稀疏环境的其他方法需要内在奖励生成器，诸如内在Currency模块（ICM）和增强Currency模块（ACM）。Caps-EM使用更紧凑的架构，无需内在奖励。使用ViZDoom进行测试，Caps-EM使用的可训练网络参数分别比ICM和深度增强Currency模块（D-ACM）少44%和83%，平均时间分别比ICM和D-ACM提高1141%和437%，以在“My Way Home”场景中收敛到策略功能。
C-STEP: Continuous Space-Time Empowerment for Physics-informed Safe Reinforcement Learning of Mobile Agents
Faculty of Technology, Bielefeld University, Germany（比勒菲尔德大学技术学院，德国）；Department of Electrical Engineering and Information Technology, Paderborn University, Germany（帕德博恩大学电气工程与信息科技系，德国）
School of Automation Science and Engineering, South China University of Technology（华南理工大学自动化科学与工程学院）；State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所机器人重点实验室）；State Key Laboratory of Robotics and System, School of Mechanical Engineering and Automation, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）机械工程与自动化学院机器人与系统重点实验室）
Human-Aware Robot Navigation via Reinforcement Learning with Hindsight Experience Replay and Curriculum Learning
The Chinese University of Hong Kong（香港中文大学）；Southern University of Science and Technology（南方科技大学）；Shenzhen Research Institute of the Chinese University of Hong Kong（香港中文大学深圳研究院）
Learning Social Navigation from Positive and Negative Demonstrations and Rule-Based Specifications
Department of Artificial Intelligence, Korea University（人工智能系，韩国大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Department of Artificial Intelligence, Yonsei University（人工智能系，yonsei大学）；Robotics Institute, School of Computer Science at Carnegie Mellon University（机器人研究所，卡内基梅隆大学计算机科学学院）；Mobinn；Department of Electrical and Computer Engineering（电气与计算机工程系）；Department of Statistics（统计学系）
