arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.01285cs.LGcs.AIstat.ML
收藏

多LLM自适应置信推理用于可靠的LLM响应

Multi-LLM Adaptive Conformal Inference for Reliable LLM Responses

Department of Applied Statistics and Data Science, Yonsei University(应用统计与数据科学系,延世大学);Department of Mathematical Sciences, KAIST(数学科学系,韩国科学技术院)

Kangjun Noh, Seongchan Lee, Ilmun Kim, Kyungwoo Song

AI总结:

多LLM自适应置信推理通过集成生成更准确的事实性分数,提升高风险领域中LLM响应的可靠性和准确性。

摘要:

确保事实性对于在医疗和法律等高风险领域安全使用大型语言模型(LLMs)至关重要。置信推理提供分布无关的保证,但现有方法要么过于保守,丢弃了许多真实声明,要么依赖于自适应误差率和简单的线性模型,无法捕捉复杂的群体结构。为了解决这些挑战,我们重新公式化置信推理为乘法过滤设置,将事实性建模为声明级别的分数乘积。我们的方法,多LLM自适应置信推理(MACI),利用集成来生成更准确的事实性分数,在我们的实验中导致了更高的保留率,同时通过群体条件校准保持有效性。实验表明,MACI一致地实现了用户指定的覆盖范围,比基线方法具有显著更高的保留率和更低的时间成本。我们的仓库可在https://github.com/MLAI-Yonsei/MACI上获取。

英文摘要:

Ensuring factuality is essential for the safe use of Large Language Models (LLMs) in high-stakes domains such as medicine and law. Conformal inference provides distribution-free guarantees, but existing approaches are either overly conservative, discarding many true-claims, or rely on adaptive error rates and simple linear models that fail to capture complex group structures. To address these challenges, we reformulate conformal inference in a multiplicative filtering setting, modeling factuality as a product of claim-level scores. Our method, Multi-LLM Adaptive Conformal Inference (MACI), leverages ensembles to produce more accurate factuality-scores, which in our experiments led to higher retention, while validity is preserved through group-conditional calibration. Experiments show that MACI consistently achieves user-specified coverage with substantially higher retention and lower time cost than baselines. Our repository is available at https://github.com/MLAI-Yonsei/MACI

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑