多任务策略的主动微调
Active Fine-Tuning of Multi-Task Policies
AI总结：
本文提出主动多任务微调算法AMF，在有限演示预算下通过选择信息增益最大的任务进行演示，以高效微调预训练策略，并提供了理论保证和实验验证。
摘要：
预训练的通用策略在机器人学习中正迅速获得重要性，因为它们有望快速适应新的领域内任务。这种适应通常依赖于为特定目标任务收集新的演示，并应用模仿学习算法，如行为克隆。然而，一旦需要学习多个任务，我们必须决定应演示哪些任务以及演示的频率。我们研究这一多任务问题，并探索一个交互式框架，其中智能体自适应地选择要演示的任务。我们提出AMF（主动多任务微调），一种在有限的演示预算下通过收集对专家策略产生最大信息增益的演示来最大化多任务策略性能的算法。我们在正则性假设下推导了AMF的性能保证，并展示了其在复杂和高维环境中高效微调神经策略的经验有效性。
英文摘要：
Pre-trained generalist policies are rapidly gaining relevance in robot learning due to their promise of fast adaptation to novel, in-domain tasks. This adaptation often relies on collecting new demonstrations for a specific task of interest and applying imitation learning algorithms, such as behavioral cloning. However, as soon as several tasks need to be learned, we must decide which tasks should be demonstrated and how often? We study this multi-task problem and explore an interactive framework in which the agent adaptively selects the tasks to be demonstrated. We propose AMF (Active Multi-task Fine-tuning), an algorithm to maximize multi-task policy performance under a limited demonstration budget by collecting demonstrations yielding the largest information gain on the expert policy. We derive performance guarantees for AMF under regularity assumptions and demonstrate its empirical effectiveness to efficiently fine-tune neural policies in complex and high-dimensional environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
