创意代理：通过想象力赋能代理以完成创意任务
Creative Agents: Empowering Agents with Imagination for Creative Tasks
AI总结：
本研究提出通过增强想象力的创意代理，首次在Minecraft生存模式中实现多样化建筑创建，利用大语言模型和扩散模型提升创造力表现。
摘要：
我们研究构建具有具身能力的代理以完成开放性创意任务。尽管现有方法构建了能够执行多样化开放性任务的指令遵循代理，但它们都无法展示创造力——即能够提供新颖且多样的解决方案的能力，这种能力隐含在语言指令中。这一限制来自于它们无法将抽象语言指令转换为具体目标，并执行针对此类复杂目标的长周期规划。鉴于人类在进行创意任务时会使用想象力，我们提出了一类解决方案，其中控制器被增强了一个生成任务结果详细想象的想象器，该想象器根据语言指令生成详细的想象。我们介绍了几种实现创意代理组件的方法。我们通过大型语言模型进行文本想象或通过扩散模型进行视觉想象来实现想象器。控制器可以是行为克隆策略或预训练的基础模型，该模型能够生成环境中的可执行代码。我们通过具有挑战性的开放世界游戏Minecraft对创意任务进行基准测试，其中代理在自由形式的语言指令下创建多样化的建筑。我们提出了一种新的评估指标用于开放性创意任务，利用GPT-4V，它在许多方面优于现有指标。我们对创意代理进行了详细的实验分析，显示创意代理是第一个在Minecraft生存模式中完成多样化建筑创建的AI代理。我们的基准和模型对未来的创意代理研究是开源的（https://github.com/PKU-RL/Creative-Agents）
英文摘要：
We study building embodied agents for open-ended creative tasks. While existing methods build instruction-following agents that can perform diverse open-ended tasks, none of them demonstrates creativity -- the ability to give novel and diverse solutions implicit in the language instructions. This limitation comes from their inability to convert abstract language instructions into concrete goals and perform long-horizon planning for such complicated goals. Given the observation that humans perform creative tasks with imagination, we propose a class of solutions, where the controller is enhanced with an imaginator generating detailed imaginations of task outcomes conditioned on language instructions. We introduce several approaches to implementing the components of creative agents. We implement the imaginator with either a large language model for textual imagination or a diffusion model for visual imagination. The controller can either be a behavior-cloning policy or a pre-trained foundation model generating executable codes in the environment. We benchmark creative tasks with the challenging open-world game Minecraft, where the agents create diverse buildings given free-form language instructions. We propose novel evaluation metrics for open-ended creative tasks utilizing GPT-4V, which holds many advantages over existing metrics. We perform a detailed experimental analysis of creative agents, showing that creative agents are the first AI agents accomplishing diverse building creation in the survival mode of Minecraft. Our benchmark and models are open-source for future research on creative agents (https://github.com/PKU-RL/Creative-Agents).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
