arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.18014cs.CLcs.LG
收藏

面向LLM结构化输出的实时可信度评分及数据提取

Real-Time Trustworthiness Scoring for LLM Structured Outputs and Data Extraction

Cleanlab

Hui Wen Goh, Jonas Mueller

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出CONSTRUCT，一种实时评估LLM结构化输出可信度的方法，通过评分帮助优先处理人类审核，支持复杂结构化输出，无需标注数据或定制模型部署。

摘要：

当前LLM的结构化输出存在随机错误，阻碍企业AI部署。本文提出CONSTRUCT，一种实时不确定性估计器，用于评分LLM结构化输出的可信度。低分输出更可能包含错误，从而自动优先处理有限的人工审核带宽。CONSTRUCT还对结构化输出中的每个字段进行评分，帮助审核人员快速识别错误部分。本文方法适用于任何LLM（包括黑盒LLM API），无需标注训练数据或定制模型部署，支持复杂结构化输出，包含异构字段和嵌套JSON模式。我们还引入了首个公开的LLM结构化输出基准，具有可靠的地面真实值。在四个数据集基准上，CONSTRUCT在各种LLM（包括Gemini 3和GPT-5）的输出中检测错误的精度/召回率显著高于现有技术。

英文摘要：

Structured Outputs from current LLMs exhibit sporadic errors, hindering enterprise AI deployment. We present CONSTRUCT, a real-time uncertainty estimator that scores the trustworthiness of LLM Structured Outputs. Lower-scoring outputs are more likely to contain errors, enabling automatic prioritization of limited human review bandwidth. CONSTRUCT additionally scores the trustworthiness of each field within a Structured Output, helping reviewers quickly identify which parts of the output are incorrect. Our method is suitable for any LLM (including black-box LLM APIs without logprobs), does not require labeled training data or custom model deployment, and supports complex Structured Outputs with heterogeneous fields and nested JSON schemas. We also introduce one of the first public LLM Structured Output benchmarks with reliable ground-truth values. Over this four-dataset benchmark, CONSTRUCT detects errors in outputs from various LLMs (including Gemini 3 and GPT-5) with significantly higher precision/recall than existing techniques.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑