arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.07006cs.AI
收藏

LLM性能预测器:在混合人机审核系统中学习何时升级

LLM Performance Predictors: Learning When to Escalate in Hybrid Human-AI Moderation Systems

Reichman University(雷赫曼大学)

Or Bachar, Or Levi, Sardhendu Mishra, Adi Levi, Manpreet Singh Minhas, Justin Miller, Omer Ben-Porat, Eilon Sheetrit, Jonathan Morra

AI总结:

本文提出了一种基于LLM性能预测器的框架,用于在混合人机审核系统中学习何时升级,通过提升准确性与成本权衡实现更高效的内容审核。

摘要:

随着LLM越来越多地融入人机协同的内容审核系统,一个核心挑战是决定何时可以信任LLM的输出,何时更应升级进行人工审核。我们提出了一种新的监督LLM不确定性量化框架,学习基于LLM性能预测器(LPPs)的专用元模型,这些LPPs源自LLM输出:对数概率、熵以及新的不确定性归因指标。我们证明了我们的方法能够在现实中的人机工作流程中实现成本敏感的分类:升级高风险案例同时自动化其余部分。在最先进的LLM上进行实验,包括现成的(Gemini、GPT)和开源的(Llama、Qwen),在多模态和多语言审核任务中,展示了在准确性与成本权衡上的显著改进。除了不确定性估计外,LPPs通过提供新的见解来增强可解释性,例如揭示失败条件(如模糊内容与不明确的政策)。本工作建立了一个以不确定性为导向、可扩展且负责任的人机审核工作流程的框架。

英文摘要:

As LLMs are increasingly integrated into human-in-the-loop content moderation systems, a central challenge is deciding when their outputs can be trusted versus when escalation for human review is preferable. We propose a novel framework for supervised LLM uncertainty quantification, learning a dedicated meta-model based on LLM Performance Predictors (LPPs) derived from LLM outputs: log-probabilities, entropy, and novel uncertainty attribution indicators. We demonstrate that our method enables cost-aware selective classification in real-world human-AI workflows: escalating high-risk cases while automating the rest. Experiments across state-of-the-art LLMs, including both off-the-shelf (Gemini, GPT) and open-source (Llama, Qwen), on multimodal and multilingual moderation tasks, show significant improvements over existing uncertainty estimators in accuracy-cost trade-offs. Beyond uncertainty estimation, the LPPs enhance explainability by providing new insights into failure conditions (e.g., ambiguous content vs. under-specified policy). This work establishes a principled framework for uncertainty-aware, scalable, and responsible human-AI moderation workflows.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑