超越幻觉：一种衡量开源大语言模型可靠性的综合评分
Beyond Hallucinations: A Composite Score for Measuring Reliability in Open-Source Large Language Models
AI总结：
本文提出综合可靠性评分（CRS）用于评估开源大语言模型的可靠性，通过实验发现最可靠的系统在准确率、鲁棒性和校准不确定性间取得平衡。
摘要：
大型语言模型（LLMs）如LLaMA、Mistral和Gemma在医疗、法律和金融等关键决策领域得到越来越多的应用，但其可靠性仍存疑。它们常常做出过度自信的错误，输入变化下性能下降，且缺乏明确的不确定性估计。现有评估碎片化，仅解决孤立方面。我们引入了综合可靠性评分（CRS），一种将校准、鲁棒性和不确定性量化整合到单一可解释指标的统一框架。通过在十个领先开源LLM上五个问答数据集上的实验，我们评估了基线、扰动和校准方法下的性能。CRS提供稳定的模型排名，揭示了单个指标无法发现的隐藏失败模式，并指出最可靠的系统在准确率、鲁棒性和校准不确定性之间达到平衡。
英文摘要：
Large Language Models (LLMs) like LLaMA, Mistral, and Gemma are increasingly used in decision-critical domains such as healthcare, law, and finance, yet their reliability remains uncertain. They often make overconfident errors, degrade under input shifts, and lack clear uncertainty estimates. Existing evaluations are fragmented, addressing only isolated aspects. We introduce the Composite Reliability Score (CRS), a unified framework that integrates calibration, robustness, and uncertainty quantification into a single interpretable metric. Through experiments on ten leading open-source LLMs across five QA datasets, we assess performance under baselines, perturbations, and calibration methods. CRS delivers stable model rankings, uncovers hidden failure modes missed by single metrics, and highlights that the most dependable systems balance accuracy, robustness, and calibrated uncertainty.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
