用LLM作为裁判评估安全度的指标
Evaluating Metrics for Safety with LLM-as-Judges
AI总结：
本文提出通过加权指标和置信度阈值来提升LLM作为裁判在关键信息流中的安全性和可靠性。
摘要：
大型语言模型（LLMs）越来越多地被用于文本处理流程中，以智能地响应各种输入和生成任务。这提出了用LLM替代瓶颈信息流中的人力角色的可能性，无论是由于人员不足还是流程复杂性。然而，LLM会犯错，且某些处理任务是安全关键的。例如，根据医院转诊信对术后患者进行分诊，或在核设施中更新工作队伍的现场访问时间表。如果我们想将LLM引入以前由人类执行的关键信息流中，如何使它们安全可靠？与其对增强生成框架或基于图的技术做出表现性声明，本文认为安全论点应聚焦于从LLM过程中评估点获得的证据类型，特别是采用LLM作为裁判（LaJ）评估器的框架。本文认为，尽管我们无法从许多自然语言处理任务中获得确定性的评估结果，但通过采用一组加权指标，可能可以降低评估中的错误风险，利用上下文敏感性定义错误严重性，并设计置信度阈值，当评估者之间的一致性较低时触发人类对关键LaJ判断的审查。
英文摘要：
LLMs (Large Language Models) are increasingly used in text processing pipelines to intelligently respond to a variety of inputs and generation tasks. This raises the possibility of replacing human roles that bottleneck existing information flows, either due to insufficient staff or process complexity. However, LLMs make mistakes and some processing roles are safety critical. For example, triaging post-operative care to patients based on hospital referral letters, or updating site access schedules in nuclear facilities for work crews. If we want to introduce LLMs into critical information flows that were previously performed by humans, how can we make them safe and reliable? Rather than make performative claims about augmented generation frameworks or graph-based techniques, this paper argues that the safety argument should focus on the type of evidence we get from evaluation points in LLM processes, particularly in frameworks that employ LLM-as-Judges (LaJ) evaluators. This paper argues that although we cannot get deterministic evaluations from many natural language processing tasks, by adopting a basket of weighted metrics it may be possible to lower the risk of errors within an evaluation, use context sensitivity to define error severity and design confidence thresholds that trigger human review of critical LaJ judgments when concordance across evaluators is low.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
