人工智能体跟踪湍流羽流的涌现行为和神经动力学
Emergent behavior and neural dynamics in artificial agents tracking turbulent plumes
摘要:
跟踪湍流羽流以定位其源是一个复杂的控制问题,因为它需要多传感器集成,并且必须对间歇性气味、改变风向和可变羽流统计数据具有鲁棒性。这一任务通常由飞行的昆虫执行,通常是长距离的,以追求食物或配偶。许多实验研究已经详细研究了这种显著行为的几个方面。在这里,我们采取了一种补充的计算机方法,使用经过强化学习训练的人工代理来开发对支持羽流跟踪的行为和神经计算的综合理解。具体来说,我们使用深度强化学习(DRL)来训练递归神经网络(RNN)代理,以定位模拟湍流羽流的来源。有趣的是,智能体的紧急行为类似于飞行昆虫的行为,RNN学会表示与任务相关的变量,例如头部方向和自上次气味遭遇以来的时间。我们的分析提出了一个有趣的实验可验证的假设,用于跟踪改变风向的羽流-代理人遵循当地羽流形状,而不是当前的风向。虽然反射性的短记忆行为足以在恒定的风中跟踪羽流,但更长时间尺度的记忆对于跟踪切换方向的羽流至关重要。在神经动力学层面,RNN的种群活动是低维的,并组织成不同的动力学结构,与行为模块有一定的对应关系。我们的计算机模拟方法为湍流羽流跟踪策略提供了关键的直觉,并激励了未来有针对性的实验和理论发展。
英文摘要:
Tracking a turbulent plume to locate its source is a complex control problem because it requires multi-sensory integration and must be robust to intermittent odors, changing wind direction, and variable plume statistics. This task is routinely performed by flying insects, often over long distances, in pursuit of food or mates. Several aspects of this remarkable behavior have been studied in detail in many experimental studies. Here, we take a complementary in silico approach, using artificial agents trained with reinforcement learning to develop an integrated understanding of the behaviors and neural computations that support plume tracking. Specifically, we use deep reinforcement learning (DRL) to train recurrent neural network (RNN) agents to locate the source of simulated turbulent plumes. Interestingly, the agents' emergent behaviors resemble those of flying insects, and the RNNs learn to represent task-relevant variables, such as head direction and time since last odor encounter. Our analyses suggest an intriguing experimentally testable hypothesis for tracking plumes in changing wind direction -- that agents follow local plume shape rather than the current wind direction. While reflexive short-memory behaviors are sufficient for tracking plumes in constant wind, longer timescales of memory are essential for tracking plumes that switch direction. At the level of neural dynamics, the RNNs' population activity is low-dimensional and organized into distinct dynamical structures, with some correspondence to behavioral modules. Our in silico approach provides key intuitions for turbulent plume tracking strategies and motivates future targeted experimental and theoretical developments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。