arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.04562cs.LGcs.CLq-bio.QM
收藏

通过迭代细化实现外推式受控序列生成

Extrapolative Controlled Sequence Generation via Iterative Refinement

New York University(纽约大学);Google DeepMind(谷歌DeepMind)

Vishakh Padmakumar, Richard Yuanzhe Pang, He He, Ankur P. Parikh

AI总结:

针对属性值超出训练范围的受控序列生成问题,本文提出 ICE,通过迭代局部编辑和小幅改进序列对训练实现外推,并在自然语言与蛋白质工程任务上显著超越现有方法。

摘要:

我们研究外推式受控生成问题,即生成属性值超出训练期间所见范围的序列。该任务在自动化设计中具有重要意义,尤其是药物发现,其目标是设计比现有序列更优(例如更稳定)的新型蛋白质。因此,根据定义,目标序列及其属性值处于训练分布之外,这给旨在直接生成目标序列的现有方法带来挑战。为此,本研究提出 Iterative Controlled Extrapolation(ICE,迭代受控外推),通过迭代方式对序列进行局部编辑以实现外推。我们在合成生成的、属性值表现出小幅提升的序列对上训练模型。在一项自然语言任务(情感分析)和两项蛋白质工程任务(ACE2 稳定性与 AAV 适应性)上的结果表明,尽管 ICE 十分简单,但其性能显著优于最先进方法。我们的代码和模型可在 https://github.com/vishakhpk/iter-extrapolation 获取。

英文摘要:

We study the problem of extrapolative controlled generation, i.e., generating sequences with attribute values beyond the range seen in training. This task is of significant importance in automated design, especially drug discovery, where the goal is to design novel proteins that are \textit{better} (e.g., more stable) than existing sequences. Thus, by definition, the target sequences and their attribute values are out of the training distribution, posing challenges to existing methods that aim to directly generate the target sequence. Instead, in this work, we propose Iterative Controlled Extrapolation (ICE) which iteratively makes local edits to a sequence to enable extrapolation. We train the model on synthetically generated sequence pairs that demonstrate small improvement in the attribute value. Results on one natural language task (sentiment analysis) and two protein engineering tasks (ACE2 stability and AAV fitness) show that ICE considerably outperforms state-of-the-art approaches despite its simplicity. Our code and models are available at: https://github.com/vishakhpk/iter-extrapolation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑