视觉上下文学习能否胜任组合式医疗任务？
Is Visual in-Context Learning for Compositional Medical Tasks within Reach?
AI总结：
本文研究视觉上下文学习在组合式医疗任务中的可行性，提出基于合成任务生成引擎的训练方法，并分析码本与掩码训练目标的作用，为多模态医疗任务序列提供见解。
摘要：
本文探讨了视觉上下文学习（visual in-context learning）的潜力，旨在使单一模型能够在测试阶段无需重新训练即可处理多种任务并适应新任务。与以往方法不同，我们的重点在于训练上下文学习器以适应任务序列，而非单个任务。我们的目标是利用单一模型解决包含多个中间步骤的复杂任务，使用户能够在测试时灵活定义完整的视觉流程。为此，我们首先分析了视觉上下文学习架构的性质与局限性，特别关注码本（codebooks）的作用。随后，我们提出了一种新方法，利用合成的组合式任务生成引擎来训练上下文学习器。该引擎从任意分割数据集引导生成任务序列，从而支持针对组合式任务的视觉上下文学习器训练。此外，我们研究了不同的基于掩码的训练目标，以深入了解如何更好地训练模型来解决复杂的组合式任务。我们的探索不仅为多模态医疗任务序列提供了重要见解，还指出了需要进一步解决的挑战。
英文摘要：
In this paper, we explore the potential of visual in-context learning to enable a single model to handle multiple tasks and adapt to new tasks during test time without re-training. Unlike previous approaches, our focus is on training in-context learners to adapt to sequences of tasks, rather than individual tasks. Our goal is to solve complex tasks that involve multiple intermediate steps using a single model, allowing users to define entire vision pipelines flexibly at test time. To achieve this, we first examine the properties and limitations of visual in-context learning architectures, with a particular focus on the role of codebooks. We then introduce a novel method for training in-context learners using a synthetic compositional task generation engine. This engine bootstraps task sequences from arbitrary segmentation datasets, enabling the training of visual in-context learners for compositional tasks. Additionally, we investigate different masking-based training objectives to gather insights into how to train models better for solving complex, compositional tasks. Our exploration not only provides important insights especially for multi-modal medical task sequences but also highlights challenges that need to be addressed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
