arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.08577cs.AIcs.LGq-bio.QM
收藏

面向毒物预测的本体预训练

Ontology Pre-training for Poison Prediction

Otto-von-Güricke University Magdeburg(马格德堡奥托·冯·格里克大学);University of Zurich(苏黎世大学);University of St. Gallen(圣加仑大学)

Martin Glauer, Fabian Neuhaus, Till Mossakowski, Janna Hastings

摘要:

将人类知识集成到神经网络中有可能提高其鲁棒性和可解释性。我们已经开发出一种新的方法来整合知识本体到一个Transformer网络的结构,我们称之为本体预训练:我们训练网络预测成员的本体类作为一种方式嵌入到网络的本体结构,并随后微调网络的特定预测任务。我们将这种方法应用于基于其分子结构预测小分子潜在毒性的案例研究,这是生命科学化学中机器学习的一项具有挑战性的任务。我们的方法改进了现有技术,而且还有几个额外的好处。首先,我们能够证明,与没有本体预训练的情况相比,模型在进行预测时学会将注意力集中在更有意义的化学基团上,从而为提高鲁棒性和可解释性铺平了道路。第二,本体预训练后的训练时间减少,表明模型在本体预训练的情况下比在没有本体预训练的情况下更好地学习毒性预测的重要性。该策略作为一种神经符号方法具有普遍适用性,可以将有意义的语义嵌入到神经网络中。

英文摘要:

Integrating human knowledge into neural networks has the potential to improve their robustness and interpretability. We have developed a novel approach to integrate knowledge from ontologies into the structure of a Transformer network which we call ontology pre-training: we train the network to predict membership in ontology classes as a way to embed the structure of the ontology into the network, and subsequently fine-tune the network for the particular prediction task. We apply this approach to a case study in predicting the potential toxicity of a small molecule based on its molecular structure, a challenging task for machine learning in life sciences chemistry. Our approach improves on the state of the art, and moreover has several additional benefits. First, we are able to show that the model learns to focus attention on more meaningful chemical groups when making predictions with ontology pre-training than without, paving a path towards greater robustness and interpretability. Second, the training time is reduced after ontology pre-training, indicating that the model is better placed to learn what matters for toxicity prediction with the ontology pre-training than without. This strategy has general applicability as a neuro-symbolic approach to embed meaningful semantics into neural networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑