arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.03190q-bio.CB
收藏

基于确定性策略梯度的集体神经嵴细胞迁移深度强化学习模型

A deep reinforcement learning model based on deterministic policy gradient for collective neural crest cell migration

Yihao Zhang, Zhaojie Chai, Yubing Sun, George Lykotrafitis

摘要:

模拟细胞相互作用,如运动的共吸引和接触抑制,是理解集体细胞迁移的关键。在这里,我们提出了一种新的深度强化学习模型,用于集体神经嵴细胞迁移。我们应用深度确定性策略梯度算法结合粒子动力学仿真环境来训练智能体以确定迁移路径,由于领导者和跟随者神经嵴细胞的迁移机制不同,我们训练了两种类型的智能体(领导者和跟随者)来学习集体细胞迁移行为.对于一个领导者代理,我们考虑一个线性组合的全球任务,导致在最短路径的目标源,和一个本地的任务,导致在协调运动沿当地的化学吸引梯度。对于一个追随者代理,我们只考虑本地任务。首先,我们证明了领导细胞学习到的自驱动力近似指向基板,这意味着代理能够学习遵循到达目标的最短路径。为了验证我们的方法,我们比较了总的时间流逝的代理达到基板使用所提出的方法和使用基于代理的模型计算的时间。使用这两种方法计算的迁移时间间隔的分布没有显着差异。然后我们研究了运动的共吸引和接触抑制对集体领导细胞迁移的影响。我们表明,整体领导细胞迁移的情况下,共同吸引力是缓慢的,因为共同吸引力减轻了源驱动的效果。此外,我们发现,领导者和追随者代理学习遵循类似的迁移行为,在实验观察。总的来说,我们提出的方法提供了有用的见解,如何应用强化学习技术来模拟集体细胞迁移。

英文摘要:

Modeling cell interactions such as co-attraction and contact-inhibition of locomotion is essential for understanding collective cell migration. Here, we propose a novel deep reinforcement learning model for collective neural crest cell migration. We apply the deep deterministic policy gradient algorithm in association with a particle dynamics simulation environment to train agents to determine the migration path. Because of the different migration mechanisms of leader and follower neural crest cells, we train two types of agents (leaders and followers) to learn the collective cell migration behavior. For a leader agent, we consider a linear combination of a global task, resulting in the shortest path to the target source, and a local task, resulting in a coordinated motion along the local chemoattractant gradient. For a follower agent, we consider only the local task. First, we show that the self-driven forces learned by the leader cell point approximately to the placode, which means that the agent is able to learn to follow the shortest path to the target. To validate our method, we compare the total time elapsed for agents to reach the placode computed using the proposed method and the time computed using an agent-based model. The distributions of the migration time intervals calculated using the two methods are shown to not differ significantly. We then study the effect of co-attraction and contact-inhibition of locomotion to the collective leader cell migration. We show that the overall leader cell migration for the case with co-attraction is slower because the co-attraction mitigates the source-driven effect. In addition, we find that the leader and follower agents learn to follow a similar migration behavior as in experimental observations. Overall, our proposed method provides useful insight on how to apply reinforcement learning techniques to simulate collective cell migration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑