arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.00863cs.CLcs.AIcs.LG
收藏

UniMoT:具有离散标记表示的统一分子-文本语言模型

UniMoT: Unified Molecule-Text Language Model with Discrete Token Representation

Tsinghua University(清华大学);Tencent AI Lab(腾讯AI实验室);Northwestern Polytechnical University(西北工业大学);Hong Kong University of Science and Technology(香港科技大学)

Shuhan Guo, Yatao Bian, Ruibing Wang, Nan Yin, Zhen Wang, Quanming Yao

AI总结:

UniMoT通过向量量化分词器将分子转化为离散标记,统一分子与文本模态,采用自回归训练,在分子理解和生成任务上达到最先进性能。

摘要:

大型语言模型(LLM)在各种任务中的显著成功,促使研究社区将其能力扩展到分子应用领域。然而,大多数分子LLM采用基于适配器的架构,这种架构未能平等对待分子和文本模态,并且缺乏针对分子模态的监督信号。为解决这些问题,我们引入了UniMoT,一个采用基于分词器架构的统一分子-文本LLM,该架构用分子标记扩展了LLM的词汇表。具体来说,我们引入了一个由向量量化驱动的分词器,该分词器结合了Q-Former以弥合分子与文本之间的模态差距。该分词器将分子转换为具有因果依赖性的分子标记序列,封装了高级分子和文本信息。借助此分词器,UniMoT能够在共享标记表示和自回归训练范式下统一分子和文本模态,使其能够将分子解释为一种外语,并像生成文本一样生成分子。遵循四阶段训练方案,UniMoT成为一个多模态通才,能够执行分子到文本和文本到分子的任务。大量实验表明,UniMoT在广泛的分子理解和生成任务中实现了最先进的性能。

英文摘要:

The remarkable success of Large Language Models (LLMs) across diverse tasks has driven the research community to extend their capabilities to molecular applications. However, most molecular LLMs employ adapter-based architectures that do not treat molecule and text modalities equally and lack a supervision signal for the molecule modality. To address these issues, we introduce UniMoT, a Unified Molecule-Text LLM adopting a tokenizer-based architecture that expands the vocabulary of LLM with molecule tokens. Specifically, we introduce a Vector Quantization-driven tokenizer that incorporates a Q-Former to bridge the modality gap between molecule and text. This tokenizer transforms molecules into sequences of molecule tokens with causal dependency, encapsulating high-level molecular and textual information. Equipped with this tokenizer, UniMoT can unify molecule and text modalities under a shared token representation and an autoregressive training paradigm, enabling it to interpret molecules as a foreign language and generate them as text. Following a four-stage training scheme, UniMoT emerges as a multi-modal generalist capable of performing both molecule-to-text and text-to-molecule tasks. Extensive experiments demonstrate that UniMoT achieves state-of-the-art performance across a wide range of molecule comprehension and generation tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑