大型结构化动作空间中的强化学习:脊髓损伤康复决策支持的仿真研究
Reinforcement learning in large, structured action spaces: A simulation study of decision support for spinal cord injury rehabilitation
AI总结:
针对脊髓损伤康复中动作空间大且训练数据有限的挑战,提出基于领域知识和嵌入技术的两种治疗方法分组策略,结合Fitted Q Iteration训练智能体,仿真证明其能有效改善治疗决策。
摘要:
强化学习(RL)已在多种应用中帮助改善了决策制定。然而,在某些应用中应用传统RL具有挑战性,例如脊髓损伤(SCI)患者的康复。除其他因素外,在该领域使用RL困难的原因在于可能的治疗方法很多(即大型动作空间)且患者很少(即有限的训练数据)。SCI的治疗具有自然的分组,因此我们提出了两种对治疗方法进行分组的方法,以便RL智能体能够从有限数据中有效学习。一种依赖于SCI康复的领域知识,另一种使用嵌入技术学习治疗方法之间的相似性。随后,我们使用Fitted Q Iteration训练一个学习最优治疗方案的智能体。通过旨在反映SCI康复特性的仿真研究,我们发现这两种方法均能帮助改善物理治疗师的治疗决策,但基于领域知识的方法表现更好。我们的研究结果提供了一个“概念验证”,表明RL可用于帮助改善SCI患者的治疗,并表明继续收集数据并将RL应用于该领域是值得的。
英文摘要:
Reinforcement learning (RL) has helped improve decision-making in several applications. However, applying traditional RL is challenging in some applications, such as rehabilitation of people with a spinal cord injury (SCI). Among other factors, using RL in this domain is difficult because there are many possible treatments (i.e., large action space) and few patients (i.e., limited training data). Treatments for SCIs have natural groupings, so we propose two approaches to grouping treatments so that an RL agent can learn effectively from limited data. One relies on domain knowledge of SCI rehabilitation and the other learns similarities among treatments using an embedding technique. We then use Fitted Q Iteration to train an agent that learns optimal treatments. Through a simulation study designed to reflect the properties of SCI rehabilitation, we find that both methods can help improve the treatment decisions of physiotherapists, but the approach based on domain knowledge offers better performance. Our findings provide a "proof of concept" that RL can be used to help improve the treatment of those with an SCI and indicates that continued efforts to gather data and apply RL to this domain are worthwhile.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。