发表机构
University of Massachusetts Amherst; New York University; IBM Research; University of Southern California（马萨诸塞大学阿默斯特分校; 纽约大学; IBM研究院; 南加利福尼亚大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
BOReFT通过学习冻结语言模型中的低维隐藏状态干预空间作为贝叶斯优化搜索域，连接离散提议与连续优化，在Semantle和分子优化任务上超越LLM基线。
AI 中文摘要
语言模型越来越多地被用作黑盒搜索的提议模型，应用范围从程序优化到分子设计。现有方法通常通过迭代提示或参数更新来改进提议，对模型搜索空间被探索的完整性和效率控制有限。连续优化方法，如贝叶斯优化，提供了一种有原则的搜索方式，但需要一个合适的操作域。为解决这一问题，我们引入了BOReFT，它在冻结的语言模型中学习一个紧凑、低维的隐藏状态干预空间，并将该空间用作贝叶斯优化的搜索域，配合外部评分函数。实验上，我们发现学习到的域跨越语义区域，并展现出支持搜索的平滑性。理论上，我们表明语义覆盖和插值控制着学习空间中可用的最佳分数，并且从该空间解码产生一个用于自适应搜索的标准随机赌博机观测模型。我们在可解释的单词搜索任务“Semantle”以及三个更真实的从头分子性质优化发现任务上评估BOReFT。与强大的LLM基线相比，BOReFT在Semantle中找到了更多隐藏目标，并且在三个分子目标中的两个上取得了更高的性质分数。因此，我们的方法为基于LLM搜索的离散提议空间与连续黑盒优化之间提供了一座有原则的新桥梁。
英文摘要
Language models are increasingly used as proposal models for black-box search, from program optimization to molecular design. Existing approaches typically improve proposals through iterative prompting or parameter updates, offering limited control over how completely and efficiently the model's search space is explored. Continuous optimization methods, such as Bayesian optimization, provide a principled way to search but require a suitable domain to operate over. To address this, we introduce BOReFT, which learns a compact, low-dimensional space of hidden-state interventions in a frozen language model, and uses this space as the search domain for Bayesian optimization with an external scoring function. Empirically, we find that the learned domain spans semantic regions and exhibits smoothness properties that support search. Theoretically, we show that semantic coverage and interpolation control the best score available in the learned space, and that decoding from this space yields a standard stochastic-bandit observation model for adaptive search. We evaluate BOReFT on the interpretable word search task "Semantle" and on three more real-world discovery tasks in de novo molecule property optimization. Compared to strong LLM baselines, BOReFT finds in Semantle a higher number of hidden targets and, on two out of three molecular objectives, achieves higher property scores. Consequently, our method provides a principled new bridge between discrete proposal spaces of LLM-based search and continuous black-box optimization.
