多模态大语言模型辅助进化搜索用于程序化控制策略
Multimodal LLM-assisted Evolutionary Search for Programmatic Control Policies
AI总结：
本研究提出多模态大语言模型辅助进化搜索方法，用于生成透明且可验证的程序化控制策略，通过结合视觉反馈与进化搜索提高策略发现效率。
摘要：
深度强化学习在控制任务中取得了显著的成功。然而，其策略通常以不透明的神经网络形式表示，难以为人所理解、验证和调试，这损害了信任并阻碍了现实世界的部署。本工作通过引入一种新的程序化控制策略发现方法，即多模态大语言模型辅助进化搜索（MLES）来解决这一挑战。MLES利用多模态大语言模型作为程序化策略生成器，结合进化搜索来自动化策略生成。它在策略生成过程中整合了基于视觉反馈的行为分析，以识别故障模式并引导有针对性的改进，从而提高策略发现的效率，并产生适应性强、与人类一致的策略。实验结果表明，MLES在两个标准控制任务中实现了与近端策略优化（PPO）相当的性能，同时提供了透明的控制逻辑和可追溯的设计过程。该方法还克服了预定义领域特定语言的局限性，促进了知识的转移和重用，并且在各种任务中具有可扩展性，显示出作为开发透明和可验证控制策略新范式的可能性。代码可在https://github.com/QingL2000/MLES上公开获取。
英文摘要：
Deep reinforcement learning has achieved impressive success in control tasks. However, its policies, represented as opaque neural networks, are often difficult for humans to understand, verify, and debug, which undermines trust and hinders real-world deployment. This work addresses this challenge by introducing a novel approach for programmatic control policy discovery, called Multimodal Large Language Model-assisted Evolutionary Search (MLES). MLES utilizes multimodal large language models as programmatic policy generators, combining them with evolutionary search to automate policy generation. It integrates visual feedback-driven behavior analysis within the policy generation process to identify failure patterns and guide targeted improvements, thereby enhancing policy discovery efficiency and producing adaptable, human-aligned policies. Experimental results demonstrate that MLES achieves performance comparable to Proximal Policy Optimization (PPO) across two standard control tasks while providing transparent control logic and traceable design processes. This approach also overcomes the limitations of predefined domain-specific languages, facilitates knowledge transfer and reuse, and is scalable across various tasks, showing promise as a new paradigm for developing transparent and verifiable control policies. Code is publicly available at https://github.com/QingL2000/MLES.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
