arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.08174cs.CLcs.AIcs.LGcs.MM
收藏

采样后识别:多模态大语言模型中风险控制与评估的通用框架

Sample then Identify: A General Framework for Risk Control and Assessment in Multimodal Large Language Models

University of Electronic Science and Technology of China(电子科技大学);Southern University of Science and Technology(南方科技大学);University of Birmingham(伯明翰大学);The University of Hong Kong(香港大学)

Qingni Wang, Tiantian Geng, Zhiyuan Wang, Teng Wang, Bo Fu, Feng Zheng

AI总结:

针对多模态大语言模型在开放环境中的可信度问题,提出通用两步框架TRON,通过保形分数采样最小响应集并基于自洽性识别高质量响应以控制错误率,实现高效的风险控制与评估。

摘要:

多模态大语言模型(MLLMs)在各类任务中展现出可喜的进展,但依然面临显著的可信度问题。先前的研究在语言建模中应用分裂保形预测(SCP)来构建具有统计保证的预测集。然而,这些方法通常依赖模型内部的logits或仅限于多选场景,这阻碍了其在动态、开放式环境中的泛化能力和适应性。本文引入了TRON,一个用于风险控制和评估的两步框架,适用于任何支持在开放式和封闭式场景下进行采样的MLLM。TRON包含两个主要组件:(1)一种新颖的保形分数,用于采样最小尺寸的响应集;(2)一种非一致性分数,基于自洽性理论识别高质量响应,通过两个特定风险水平控制错误率。此外,我们首次研究了开放式背景下预测集中的语义冗余,提出了一种基于平均集合大小的MLLM评估指标。我们在四个视频问答(VideoQA)数据集上利用八个MLLM进行的全面实验表明,TRON实现了受限于两个用户指定风险水平的理想错误率。此外,去重后的预测集在保持适应性的同时,在不同风险水平下的风险评估更加高效和稳定。

英文摘要:

Multimodal Large Language Models (MLLMs) exhibit promising advancements across various tasks, yet they still encounter significant trustworthiness issues. Prior studies apply Split Conformal Prediction (SCP) in language modeling to construct prediction sets with statistical guarantees. However, these methods typically rely on internal model logits or are restricted to multiple-choice settings, which hampers their generalizability and adaptability in dynamic, open-ended environments. In this paper, we introduce TRON, a two-step framework for risk control and assessment, applicable to any MLLM that supports sampling in both open-ended and closed-ended scenarios. TRON comprises two main components: (1) a novel conformal score to sample response sets of minimum size, and (2) a nonconformity score to identify high-quality responses based on self-consistency theory, controlling the error rates by two specific risk levels. Furthermore, we investigate semantic redundancy in prediction sets within open-ended contexts for the first time, leading to a promising evaluation metric for MLLMs based on average set size. Our comprehensive experiments across four Video Question-Answering (VideoQA) datasets utilizing eight MLLMs show that TRON achieves desired error rates bounded by two user-specified risk levels. Additionally, deduplicated prediction sets maintain adaptiveness while being more efficient and stable for risk assessment under different risk levels.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑