发表机构
The Hong Kong Polytechnic University; The Chinese University of Hong Kong; Shenzhen Loop Area Institute; Harbin Institute of Technology（香港理工大学; 香港中文大学; 深圳河套学院; 哈尔滨工业大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文针对微型机器人DRL导航训练耗时久的问题，提出含全向量化模拟器与TSR奖励框架的学习框架，将训练缩至10分钟内，支持零样本部署，可缩短设计周期、加速部署。
AI 中文摘要
微型机器人在各类应用中具有巨大潜力，而定向导航是其基本需求。深度强化学习（DRL）是近期实现微型机器人完全自主导航的强大范式，但当前基于DRL的方法对学习效率与有效性关注有限，模型训练需耗时数小时至数天，这阻碍了快速实际部署与参数优化。为解决这些挑战，本文提出一种可在数分钟内训练出有效微型机器人导航策略的学习框架。该框架中，我们开发了具备超10000个人工血管环境的全向量化模拟器，在数千个环境中并行化动力学、类激光雷达感知与可行性检查，实现约每秒190000个状态转移。为在快速训练中实现有效性，我们提出任务塑形正则化（TSR）奖励框架，该框架可加速收敛、提升最终性能，在所有评估场景中使动作变化至少降低33.7%、障碍物通过率至少提升2.1%。结果表明，该学习框架将训练时间缩短至10分钟以内，同时支持在不同微型机器人类型与导航场景中的零样本部署，总体而言，该框架可大幅缩短设计周期并加速自主微型机器人的部署。
英文摘要
Microrobots hold significant potential for various applications, where targeted navigation is a basic requirement. Deep reinforcement learning (DRL) has recently emerged as a powerful paradigm for fully autonomous microrobot navigation. Yet, current DRL-based approaches pay limited attention to learning efficiency and effectiveness, requiring hours to days for model training. Consequently, this impedes both rapid practical deployment and parameter optimization. To address these challenges, we present a learning framework that enables effective microrobot navigation policies to be trained within minutes. In the proposed framework, we develop a fully vectorized simulator with more than 10,000 artificial vascular environments, parallelizing dynamics, LiDAR-inspired perception, and feasibility checks across thousands of environments to achieve roughly 190,000 transitions per second. To achieve effectiveness in fast training, we propose a task-shaping-regularization (TSR) reward framework. The TSR framework accelerates convergence, improves final performance, reduces action variation by at least 33.7%, and increases obstacle clearance by at least 2.1% across all evaluated scenarios. Results show that the proposed learning framework reduces training time to under 10 minutes, while supporting zero-shot deployment across distinct microrobot types and navigation scenarios. Collectively, this framework can substantially shorten the design loop and accelerate the deployment of autonomous microrobots.
