可信度度量融合：提升跨领域的LLM可信度评估
Faithfulness metric fusion: Improving the evaluation of LLM trustworthiness across domains
AI总结：
通过融合多种可信度度量，提升LLM在不同领域中的可信度评估能力，并通过同质化数据集验证其有效性。
摘要：
我们提出了一种方法，用于提高大型语言模型（LLM）的准确性评估。该方法基于将基本的可信度度量合并为一个综合（融合）度量，目的是提高LLM输出的可信度。所提出的度量融合策略部署了基于树的模型来确定每个度量的重要性，该模型由人类判断评估LLM响应的可信度的整合所驱动。该融合度量被证明在所有测试领域中与人类判断的相关性更强。提高评估LLM可信度的能力，使可以更自信地依赖模型，从而在更广泛的情境中实施。此外，我们对问答和基于对话的领域中的数据集进行了同质化处理，并在该数据集中实施了人类判断和LLM响应，从而允许在跨领域中重现和测试可信度评估。
英文摘要：
We present a methodology for improving the accuracy of faithfulness evaluation in Large Language Models (LLMs). The proposed methodology is based on the combination of elementary faithfulness metrics into a combined (fused) metric, for the purpose of improving the faithfulness of LLM outputs. The proposed strategy for metric fusion deploys a tree-based model to identify the importance of each metric, which is driven by the integration of human judgements evaluating the faithfulness of LLM responses. This fused metric is demonstrated to correlate more strongly with human judgements across all tested domains for faithfulness. Improving the ability to evaluate the faithfulness of LLMs, allows for greater confidence to be placed within models, allowing for their implementation in a greater diversity of scenarios. Additionally, we homogenise a collection of datasets across question answering and dialogue-based domains and implement human judgements and LLM responses within this dataset, allowing for the reproduction and trialling of faithfulness evaluation across domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
