arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.08802cs.LG
收藏

主动流扩展用于分布外发现:从理论到分子

Active Flow Expansion for Out-of-Distribution Discovery: from Theory to Molecules

ETH Zurich(苏黎世联邦理工学院);ETH AI Center(ETH AI 中心);University of Pennsylvania(宾夕法尼亚大学);Caltech(加州理工学院);FutureHouse

Riccardo De Santi, Bruce Lee, Cristian Perez Jensen, Kimon Protopapas, Sophia Tang, Cheng-Hao Liu, Pranam Chatterjee, Yisong Yue, Andreas Krause

AI总结:

提出Active Flow Expansion (ActFlow)方法,通过验证器反馈和主动探索扩展预训练流模型的生成集,覆盖更多有效设计空间,理论证明统计学习保证,在分子和蛋白质任务上优于现有方法。

摘要:

标准流和扩散预训练匹配可用数据(例如分子)的分布,这通常只覆盖有效设计空间的一小部分。然而,在生成发现中,目标是采样有效的新自然设计,这些设计在标准模型下被赋予可忽略的概率,因此无法从拟合观测数据的标准模型中获取。为克服这一限制,我们偏离数据分布匹配,通过生成集(模型以非可忽略概率覆盖的区域)来审视生成模型。这允许引入一种新的分布外流建模学习原则:扩大模型的生成集以增加对有效设计空间的覆盖。我们提出主动流扩展(ActFlow),一种持续预训练方法,利用验证器反馈,通过迭代适应在学习的流表示中主动探索生成的合成数据,将预训练模型扩展到新的有效区域。理论上,我们建立了据我们所知首个分布外流建模的统计学习保证,将生成集扩展分析为在学习表示上的局部到全局可达过程。实验上,我们使用合适的分布外生成建模指标,在小有机分子、中等大小药物样分子、治疗性肽和蛋白质序列设计任务上评估ActFlow。结果表明,ActFlow将有效覆盖扩展到远超初始预训练模型建模的区域,显著优于广泛采用的合成流预训练方法。

英文摘要:

Standard flow and diffusion pre-training matches the distribution of available data (e.g., molecules), which often covers only a small fraction of the valid design space. In generative discovery, however, one aims to sample valid new-to-nature designs, assigned negligible probability under, and thus inaccessible to, standard models fitted to the observed data. To overcome this limitation, we depart from data distribution matching and view a generative model through its generable set: the region it covers with non-negligible probability. This allows to introduce a new learning principle for out-of-distribution flow modeling: enlarging a model's generable set to increase coverage of the valid design space. We propose Active Flow Expansion (ActFlow), a continued pre-training method that employs verifier feedback to expand a pre-trained model over new valid regions by iteratively adapting to synthetic data generated through active exploration in the learned flow representation. Theoretically, we establish to our knowledge first-of-their-kind statistical learning guarantees for out-of-distribution flow modeling, analyzing generable set expansion as a local-to-global reachability process over a learned representation. Empirically, we assess ActFlow with suitable out-of-distribution generative modeling metrics across small organic molecules, mid-sized drug-like molecules, therapeutic peptides, and protein sequence design tasks. Results show that ActFlow expands valid coverage far beyond the region modeled by the initial pre-trained model, significantly outperforming widely adopted synthetic flow pre-training methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑