发表机构
Knowledge Media Institute, The Open University, Milton Keynes, UK; Department of Business and Law, University of Milano-Bicocca, Milan, IT（开放大学知识媒体研究所; 米兰-比科卡大学商业与法律系）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文评估五个小型开源LLMs识别生物医学概念语义关系的性能，引入MeSH-Rel-4K数据集并分析三种策略，发现针对性微调使平均F1分数显著提高，突破推理瓶颈，为构建生物医学本体提供准确自动化方法。
AI 中文摘要
本体和分类法等知识组织系统对构建科学知识至关重要，但人工策划是知识管理的瓶颈。大语言模型（LLMs）为自动本体生成提供了可扩展机制，但其对复杂领域特定语义的分类能力需系统评估。本文评估了五个小型开源LLMs（参数达90亿）识别生物医学概念语义关系的性能。为此引入了MeSH-Rel-4K数据集，分析了标准提示、思维链提示和微调三种策略。结果表明针对性微调使平均F1分数提高34.1个百分点，证实直接微调有效突破较小LLMs的推理瓶颈，为构建和发展专业生物医学本体提供了准确自动化方法。
英文摘要
Knowledge Organization Systems like Ontologies and taxonomies are fundamental for structuring scientific knowledge, yet their manual curation presents a persistent bottleneck in knowledge management. While Large Language Models (LLMs) offer a scalable mechanism for automated ontology generation, their capacity to classify complex, domain-specific semantics requires systematic evaluation. In this paper, we assess the performance of five small, open-source LLMs (up to 9 billion parameters) in identifying semantic relationships between biomedical concepts. To support this evaluation, we introduce MeSH-Rel-4K, a dataset comprising 4K semantic relationships extracted from the Medical Subject Headings (MeSH). We analyse three adaptation strategies: standard prompting, Chain-of-Thought prompting, and fine-tuning. While parameter-constrained models traditionally struggle with the nuances of in-context logic, our results reveal that targeted fine-tuning increases the average F1-score by 34.1 percentage points. These results confirm that direct fine-tuning effectively exceeds the reasoning bottlenecks of smaller LLMs, providing an accurate, automated methodology for the construction and evolution of specialised biomedical ontologies.
