arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.03547cs.LGcs.AIcs.NE
收藏

立场论文:利用基础模型进行黑盒优化

Position: Leverage Foundational Models for Black-Box Optimization

Google DeepMind(谷歌DeepMind);Sakana AI

Xingyou Song, Yingtao Tian, Robert Tjarko Lange, Chansoo Lee, Yujin Tang, Yutian Chen

AI总结:

本文提出以基于序列的基础模型重塑黑盒优化,梳理相关文献,并阐述利用LLM的文本知识、Transformers建模能力和搜索空间泛化预测来改进优化的路径。

摘要:

不可否认,Large Language Models(LLM,大语言模型)在机器学习研究领域掀起了非凡的创新浪潮,并在强化学习、机器人学和计算机视觉等多个领域产生了重大影响。它们的融入迅速且具有变革性,标志着机器学习研究领域的一次重大范式转变。然而,以黑盒优化为基础的实验设计领域,受到这一范式转变的影响要小得多,尽管将LLM与优化相结合呈现出一个独特且极具探索价值的局面。在这篇立场论文中,我们围绕基于序列的基础模型来构建黑盒优化领域的框架,并梳理它们与以往文献之间的关系。我们讨论了基础语言模型能够革新优化的最有前景的方式,包括利用自由形式文本中蕴含的丰富信息来增强任务理解,利用Transformers等高度灵活的序列模型来设计更优的优化策略,以及改进在以往未见过的搜索空间上的性能预测。

英文摘要:

Undeniably, Large Language Models (LLMs) have stirred an extraordinary wave of innovation in the machine learning research domain, resulting in substantial impact across diverse fields such as reinforcement learning, robotics, and computer vision. Their incorporation has been rapid and transformative, marking a significant paradigm shift in the field of machine learning research. However, the field of experimental design, grounded on black-box optimization, has been much less affected by such a paradigm shift, even though integrating LLMs with optimization presents a unique landscape ripe for exploration. In this position paper, we frame the field of black-box optimization around sequence-based foundation models and organize their relationship with previous literature. We discuss the most promising ways foundational language models can revolutionize optimization, which include harnessing the vast wealth of information encapsulated in free-form text to enrich task comprehension, utilizing highly flexible sequence models such as Transformers to engineer superior optimization strategies, and enhancing performance prediction over previously unseen search spaces.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑