SciTrust 2.0：评估科学应用中大语言模型可信度的综合框架
SciTrust 2.0: A Comprehensive Framework for Evaluating Trustworthiness of Large Language Models in Scientific Applications
AI总结：
本文提出 SciTrust 2.0 框架，从真实性、对抗鲁棒性、安全性和伦理四个维度评估科学应用中的大语言模型，发现通用工业模型总体优于科学专用模型，并揭示了科学专用模型在高风险领域的安全与伦理推理缺陷。
摘要：
大语言模型（LLMs）在科学研究中展现了变革性潜力，但其在高风险环境中的部署引发了显著的可信度担忧。在此，我们提出 SciTrust 2.0，一个用于评估科学应用中 LLM 可信度的综合框架，涵盖四个维度：真实性、对抗鲁棒性、科学安全性和科学伦理。该框架包含通过经验证的反思微调流程和专家验证开发的新型开放式真实性基准，以及一个涵盖双重用途研究和偏见等八个子类别的科学研究新型伦理基准。我们使用准确率、语义相似度度量和基于 LLM 的评分等多种评估指标，评估了七个知名 LLM，包括四个科学专用模型和三个通用工业模型。通用工业模型在每个可信度维度上总体优于科学专用模型，其中 GPT-o4-mini 在真实性评估和对抗鲁棒性方面表现出色。科学专用模型在逻辑和伦理推理能力上表现出显著缺陷，并在安全评估中存在令人担忧的漏洞，特别是在生物安全和化学武器等高风险领域。通过开源我们的框架，我们为开发更可信的 AI 系统和推进科学背景下模型安全与伦理研究奠定了基础。
英文摘要：
Large language models (LLMs) have demonstrated transformative potential in scientific research, yet their deployment in high-stakes contexts raises significant trustworthiness concerns. Here, we introduce SciTrust 2.0, a comprehensive framework for evaluating LLM trustworthiness in scientific applications across four dimensions: truthfulness, adversarial robustness, scientific safety, and scientific ethics. Our framework incorporates novel, open-ended truthfulness benchmarks developed through a verified reflection-tuning pipeline and expert validation, alongside a novel ethics benchmark for scientific research contexts covering eight subcategories including dual-use research and bias. We evaluated seven prominent LLMs, including four science-specialized models and three general-purpose industry models, using multiple evaluation metrics including accuracy, semantic similarity measures, and LLM-based scoring. General-purpose industry models overall outperformed science-specialized models across each trustworthiness dimension, with GPT-o4-mini demonstrating superior performance in truthfulness assessments and adversarial robustness. Science-specialized models showed significant deficiencies in logical and ethical reasoning capabilities, along with concerning vulnerabilities in safety evaluations, particularly in high-risk domains such as biosecurity and chemical weapons. By open-sourcing our framework, we provide a foundation for developing more trustworthy AI systems and advancing research on model safety and ethics in scientific contexts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
