利用时间记忆的Q-learning在湍流中导航
Q-learning with temporal memory to navigate turbulence
AI总结:
针对湍流环境中的气味搜索问题,提出一种结合时间记忆的强化学习算法,利用少量可解释的嗅觉状态学习导航,发现类似飞虫的侧风恢复策略,且对环境变化具有鲁棒性。
摘要:
我们考虑了在湍流环境中的气味搜索问题。我们专注于仅对气味刺激做出反应的智能体,这些智能体无法获取空间感知信息,也没有关于气味的先验信息。我们探究了在序贯决策框架内能否稳健地学习导航至目标。我们开发了一种强化学习算法,使用少量可解释的嗅觉状态,并用真实的湍流气味线索对其进行训练。通过引入时间记忆,我们证明了气味轨迹的两个显著特征(被离散化为少数嗅觉状态)足以在真实的气味羽流中学习导航。性能取决于湍流气味的稀疏性。存在一种最佳记忆,它会忽略羽流内的空白,并在羽流外激活恢复策略。我们通过让智能体学习其恢复策略获得了最佳性能,并表明该策略主要是进行侧风投射,类似于在飞虫中观察到的行为。这种最佳策略对气味羽流的重大变化具有鲁棒性,这表明微调参数可能足以适应不同的环境。
英文摘要:
We consider the problem of olfactory searches in a turbulent environment. We focus on agents that respond solely to odor stimuli, with no access to spatial perception nor prior information about the odor. We ask whether navigation to a target can be learned robustly within a sequential decision making framework. We develop a reinforcement learning algorithm using a small set of interpretable olfactory states and train it with realistic turbulent odor cues. By introducing a temporal memory, we demonstrate that two salient features of odor traces, discretized in few olfactory states, are sufficient to learn navigation in a realistic odor plume. Performance is dictated by the sparse nature of turbulent odors. An optimal memory exists which ignores blanks within the plume and activates a recovery strategy outside the plume. We obtain the best performance by letting agents learn their recovery strategy and show that it is mostly casting cross wind, similar to behavior observed in flying insects. The optimal strategy is robust to substantial changes in the odor plumes, suggesting minor parameter tuning may be sufficient to adapt to different environments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。