使用coresets进行医学影像终身深度强化学习的选择性经验回放压缩
Selective experience replay compression using coresets for lifelong deep reinforcement learning in medical imaging
AI总结：
提出一种保持奖励分布的coreset压缩技术，用于压缩终身深度强化学习中的选择性经验回放缓冲区，在医学影像定位任务中验证了其在大幅减少存储的同时能保持模型性能。
摘要：
选择性经验回放（Selective experience replay）是将终身学习与深度强化学习相结合的一种流行策略。选择性经验回放旨在重述从先前任务中选取的经验，以避免灾难性遗忘。此外，基于选择性经验回放的技术与模型无关，并允许在不同模型间共享经验。然而，存储来自所有先前任务的经验使得使用选择性经验回放的终身学习在计算上非常昂贵，并且随着任务数量的增加而变得不切实际。为此，我们提出了一种保持奖励分布的coreset压缩技术，用于压缩为选择性经验回放存储的经验回放缓冲区（ERB）。我们在脑肿瘤分割（BRATS）数据集上评估了该coreset压缩技术，用于脑室定位任务；并在全身MRI上评估了其用于左膝盖骨、左肾、右大转子、左肺和脾脏定位的表现。在10个不同脑部MR成像环境序列上训练的coreset终身学习模型，在10倍压缩比下表现出优异的脑室定位性能，平均像素误差距离为12.93。相比之下，传统的终身学习模型定位脑室时的平均像素距离为10.87。类似地，在全身MRI上训练的coreset终身学习模型显示，在所有解剖标志点上，10倍压缩的coreset终身学习模型与传统终身学习模型之间没有显著差异（p=0.28）。所有标志点上，10倍压缩模型的平均像素距离为25.30，而传统终身学习模型为19.24。我们的结果表明，基于coreset的ERB压缩方法在压缩经验方面具有潜力，且性能没有显著下降。
英文摘要：
Selective experience replay is a popular strategy for integrating lifelong learning with deep reinforcement learning. Selective experience replay aims to recount selected experiences from previous tasks to avoid catastrophic forgetting. Furthermore, selective experience replay based techniques are model agnostic and allow experiences to be shared across different models. However, storing experiences from all previous tasks make lifelong learning using selective experience replay computationally very expensive and impractical as the number of tasks increase. To that end, we propose a reward distribution-preserving coreset compression technique for compressing experience replay buffers stored for selective experience replay. We evaluated the coreset compression technique on the brain tumor segmentation (BRATS) dataset for the task of ventricle localization and on the whole-body MRI for localization of left knee cap, left kidney, right trochanter, left lung, and spleen. The coreset lifelong learning models trained on a sequence of 10 different brain MR imaging environments demonstrated excellent performance localizing the ventricle with a mean pixel error distance of 12.93 for the compression ratio of 10x. In comparison, the conventional lifelong learning model localized the ventricle with a mean pixel distance of 10.87. Similarly, the coreset lifelong learning models trained on whole-body MRI demonstrated no significant difference (p=0.28) between the 10x compressed coreset lifelong learning models and conventional lifelong learning models for all the landmarks. The mean pixel distance for the 10x compressed models across all the landmarks was 25.30, compared to 19.24 for the conventional lifelong learning models. Our results demonstrate that the potential of the coreset-based ERB compression method for compressing experiences without a significant drop in performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
