arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.03007cs.CVcs.AIcs.LG
收藏

VOILA:基于信息价值的保真度选择框架用于成本感知的多模态问答

VOILA: Value-of-Information Guided Fidelity Selection for Cost-Aware Multimodal Question Answering

Rahul Atul Bhope, K. R. Jayaram, Vinod Muthusamy, Ritesh Kumar, Vatche Isahagian, Nalini Venkatasubramanian

AI总结:

VOILA通过基于信息价值的自适应保真度选择,在多模态问答中实现成本优化与高精度的平衡。

摘要:

尽管从检索和处理高保真度视觉输入中会产生显著成本,但大多数多模态视觉-语言系统在固定保真度水平上运行。我们引入了VOILA,这是一个基于信息价值驱动的自适应保真度选择框架,用于视觉问答(VQA)中,在模型执行前优化应检索的信息。给定一个查询,VOILA使用一个两阶段流程:一个梯度提升回归器仅基于问题特征估计每个保真度的正确性可能性,然后一个等比校准器对这些概率进行细化以实现可靠的决策。系统选择在预测准确性和检索成本下最大化预期效用的最低成本保真度。我们评估了VOILA在三个部署场景中使用五个数据集(VQA-v2、GQA、TextVQA、LoCoMo、FloodNet)和六个视觉-语言模型(VLMs)(参数从7B到235B)的性能。VOILA在各种查询类型和模型架构上一致实现了50-60%的成本降低,同时保留了90-95%的全分辨率准确性,证明了在资源受限情况下,预检索保真度选择对于优化多模态推理至关重要。

英文摘要:

Despite significant costs from retrieving and processing high-fidelity visual inputs, most multimodal vision-language systems operate at fixed fidelity levels. We introduce VOILA, a framework for Value-Of-Information-driven adaptive fidelity selection in Visual Question Answering (VQA) that optimizes what information to retrieve before model execution. Given a query, VOILA uses a two-stage pipeline: a gradient-boosted regressor estimates correctness likelihood at each fidelity from question features alone, then an isotonic calibrator refines these probabilities for reliable decision-making. The system selects the minimum-cost fidelity maximizing expected utility given predicted accuracy and retrieval costs. We evaluate VOILA across three deployment scenarios using five datasets (VQA-v2, GQA, TextVQA, LoCoMo, FloodNet) and six Vision-Language Models (VLMs) with 7B-235B parameters. VOILA consistently achieves 50-60% cost reductions while retaining 90-95% of full-resolution accuracy across diverse query types and model architectures, demonstrating that pre-retrieval fidelity selection is vital to optimize multimodal inference under resource constraints.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑