用于疟疾药物发现的大型语言模型的严格评估：性能、规模与资源效用之间的权衡
Rigorous Evaluation of Large Language Models for Malaria Drug Discovery: Trade-offs in Performance, Scale, and Resource Utility
浏览论文内容
中文总结 AI 辅助
本研究构建了Malaria-Instruct数据集，评估了多款开源LLM在疟疾虚拟筛选中的表现，发现微调后的开源LLM性能优于经典ML模型和专有模型，是高效的抗疟药物发现范式。
中文摘要 AI 辅助
我们推出Malaria-Instruct，这是一个源自ChEMBL Legacy疟疾语料库的、用于疟疾虚拟筛选的精选指令遵循数据集，并在严格的分布外数据划分下，对五个开源大型语言模型（LLM）进行了系统评估，分别是Gemma-2 2B/9B、TxGemma-2B/9B和LlaSMol-Mistral-7B。在少样本条件下，我们将其性能与经典机器学习模型（随机森林、XGBoost）以及前沿专有模型（Gemini 2.5、OpenAI o3）进行了基准对比。经过微调的LLM显著优于所有基线模型：TxGemma-9B取得了最高的ROC-AUC值（0.731±0.005），LlaSMol-Mistral-7B则拥有最佳的富集因子（EF@1%≈4.99）。领域特定微调被证明是绝对必要的，在其最佳少样本条件下，TxGemma-9B的ROC-AUC从0.731降至0.499，且Gemini 2.5（ROC-AUC≈0.53）和o3（ROC-AUC≈0.59）在未经过微调时均无法实现可靠的区分。在同等规模下，生物医学预训练带来了可衡量的优势，而化学感知预训练则产生了更出色的前瞻性富集效果。经过微调的开源LLM为抗疟虚拟筛选（VS）提供了一种引人注目的、资源高效的范式，在具有挑战性的结构条件下，其表现优于经典流程和专有推理模型。
英文摘要
We introduce Malaria-Instruct, a curated instruction-following dataset derived from the ChEMBL Legacy Malaria corpus for Malaria virtual screening, and conduct a systematic evaluation of five open-source LLMs; Gemma-2 2B/9B, TxGemma-2B/9B, and LlaSMol-Mistral-7B, on a rigorous out-of-distribution data split. Performance was benchmarked against classical ML models (Random Forest, XGBoost) and frontier proprietary models (Gemini 2.5, OpenAI o3) under few-shot conditions. Fine-tuned LLMs substantially outperformed all baselines: TxGemma-9B achieved the highest ROC-AUC ($0.731 \pm 0.005$) and LlaSMol-Mistral-7B the best enrichment factor (EF@1\% $\approx$ 4.99). Domain-specific fine-tuning proved categorically indispensable with TxGemma-9B collapsing from ROC-AUC 0.731 to 0.499, under its best few-shot condition, and neither Gemini 2.5 (ROC-AUC $\approx$ 0.53) nor o3 (ROC-AUC $\approx$ 0.59) achieved reliable discrimination without fine-tuning. Biomedical pretraining conferred a measurable advantage at equivalent scale, while chemistry-aware pretraining yielded superior prospective enrichment. Fine-tuned open-source LLMs represent a compelling, resource-efficient paradigm for antimalarial VS, outperforming both classical pipelines and proprietary reasoning models under structurally challenging conditions.
发表机构
- Magami Open Sciences Initiative（马加米开放科学倡议）
机构由 AI 辅助整理，请以论文原文为准。
