发表机构
UCL（伦敦大学学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究探究持续强化学习中任务相似性与灾难性遗忘的关系，采用Q-learning在图任务上实验，发现二者存在复杂动态关系，未证实任务相似性独立影响遗忘，需进一步研究。
AI 中文摘要
本研究探讨强化学习中任务相似性与灾难性遗忘之间的关系。灾难性遗忘是机器学习中出现的、模型会丧失对先前任务有效执行能力的现象，是持续学习的重大阻碍。本研究旨在理解新任务的相似性对先前任务性能的影响程度。研究采用可解释强化学习中的Q-learning算法，在基于图的任务上进行实验，目标是最小化到达目标的步数。研究考察了在不同相对复杂度的新任务上训练后，模型在先前学习任务上的性能表现。实验结果揭示了任务相似性与遗忘之间存在复杂的动态关系，在不同任务相似程度和任务复杂度下，遗忘严重程度出现显著波动，表明遗忘与任务的相似性和复杂度存在相互依赖关系。同时，研究还观察到遗忘程度存在较高的变异性，且任务相似性度量的分布不均。这些变量之间的关系仍不明确，在持续强化学习中，未发现任务相似性独立对遗忘产生影响的统计显著性证据。未来需开展进一步研究，以全面理解任务相似性与灾难性遗忘之间可能存在的相互作用。
英文摘要
This work explores the relationship between task similarity and catastrophic forgetting in reinforcement learning. Catastrophic forgetting, the phenomenon in machine learning of losing the ability to effectively perform on previous tasks, is a significant impediment to continual learning. This study aims to understand the extent to which the similarity of a new task influences the performance on the previous task. Interpretable reinforcement learning, specifically Q-learning, is employed on graph-based tasks with the objective of minimising the number of steps to reach a goal. The study investigates the performance on a previously learned task after training on a new task, for tasks of varying relative levels of complexity. The experimental results reveal a complex dynamic between task similarity and forgetting, with significant fluctuations in forgetting severity observed across degrees of task similarities and task complexities, and are suggestive of an interdependence of forgetting on the similarity and complexity of tasks. The observations were accompanied by observations of high degrees of variability in forgetting and an uneven distribution of task similarity measures. The relationship between these variables remains unclear and no evidence of statistical significance that task similarity has an effect, independently, on forgetting is found in continual reinforcement learning. Further research is warranted to gain a comprehensive understanding of the potential interplay between task similarity and catastrophic forgetting.
CommentsMSc thesis, University College London (UCL), 2023
