AOP-Smart：一种增强检索的大型语言模型框架用于不良后果路径分析
AOP-Smart: A RAG-Enhanced Large Language Model Framework for Adverse Outcome Pathway Analysis
AI总结：
本文提出AOP-Smart框架，通过检索增强生成技术提升大型语言模型在不良后果路径分析中的可靠性与准确性，实验表明其显著缓解了模型幻觉问题。
摘要：
不良后果路径（AOPs）是毒理学研究和风险评估中的重要知识框架。近年来，大型语言模型（LLMs）逐渐应用于AOP相关的问答和机理推理任务。然而，由于存在幻觉问题，即模型可能生成与事实不一致或缺乏证据的内容，其可靠性仍然有限。为此，本研究提出了一种面向AOP的检索增强生成（RAG）框架，即AOP-Smart。基于AOP-Wiki的官方XML数据，该方法利用关键事件（KEs）、关键事件关系（KERs）和特定AOP信息来检索相关知识以回答用户问题，从而提高大型语言模型生成结果的可靠性。为了评估所提方法的有效性，本研究构建了一个包含20个AOP相关问答任务的测试集，涵盖KE识别、上游和下游KE检索以及复杂AOP检索任务。实验在三个主流大型语言模型（Gemini、DeepSeek和ChatGPT）上进行，并在无RAG和有RAG两种设置下进行了对比测试。实验结果表明，不使用RAG时，GPT、DeepSeek和Gemini的准确率分别为15.0%、35.0%和20.0%；使用RAG后，其准确率分别提高到95.0%、100.0%和95.0%。结果表明，AOP-Smart能显著缓解大型语言模型在AOP知识任务中的幻觉问题，并大幅提高其回答的准确性和一致性。
英文摘要：
Adverse Outcome Pathways (AOPs) are an important knowledge framework in toxicological research and risk assessment. In recent years, large language models (LLMs) have gradually been applied to AOP-related question answering and mechanistic reasoning tasks. However, due to the existence of the hallucination problem, that is, the model may generate content that is inconsistent with facts or lacks evidence, their reliability is still limited. To address this issue, this study proposes an AOP-oriented Retrieval-Augmented Generation (RAG) framework, AOP-Smart. Based on the official XML data from AOP-Wiki, this method uses Key Events (KEs), Key Event Relationships (KERs), and specific AOP information to retrieve relevant knowledge for user questions, thereby improving the reliability of the generated results of large language models. To evaluate the effectiveness of the proposed method, this study constructed a test set containing 20 AOP-related question answering tasks, covering KE identification, upstream and downstream KE retrieval, and complex AOP retrieval tasks. Experiments were conducted on three mainstream large language models, Gemini, DeepSeek, and ChatGPT, and comparative tests were performed under two settings: without RAG and with RAG. The experimental results show that, without using RAG, the accuracies of GPT, DeepSeek, and Gemini were 15.0\%, 35.0\%, and 20.0\%, respectively; after using RAG, their accuracies increased to 95.0\%, 100.0\%, and 95.0\%, respectively. The results indicate that AOP-Smart can significantly alleviate the hallucination problem of large language models in AOP knowledge tasks, and greatly improve the accuracy and consistency of their answers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
