通过分层核心集选择推进复杂广域场景理解
Advancing Complex Wide-Area Scene Understanding with Hierarchical Coresets Selection
AI总结：
针对VLM难以适应未见复杂广域场景的问题，本文提出即插即用的分层核心集选择机制HCS，依据具理论保证的重要性函数筛选区域，无需微调即可提升多任务场景理解性能。
摘要：
场景理解是计算机视觉的核心任务之一，旨在从图像中提取语义信息，以识别物体、场景类别及其相互关系。尽管视觉-语言模型（VLM）的进展推动了该领域的发展，但现有VLM在适应未见的复杂广域场景方面仍面临挑战。为应对这些挑战，本文提出一种分层核心集选择（HCS）机制，以推进VLM在复杂广域场景理解中的适应。该机制基于所提出的具有理论保证的重要性函数逐步细化所选区域，该函数综合考虑效用、代表性、鲁棒性和协同性。无需额外微调，HCS使VLM能够利用最少的可解释区域快速理解任意规模的未见场景，同时缓解特征密度不足的问题。HCS是一种即插即用方法，可兼容任何VLM。实验表明，HCS在各类任务中均取得优越性能和普适性。
英文摘要：
Scene understanding is one of the core tasks in computer vision, aiming to extract semantic information from images to identify objects, scene categories, and their interrelationships. Although advancements in Vision-Language Models (VLMs) have driven progress in this field, existing VLMs still face challenges in adaptation to unseen complex wide-area scenes. To address the challenges, this paper proposes a Hierarchical Coresets Selection (HCS) mechanism to advance the adaptation of VLMs in complex wide-area scene understanding. It progressively refines the selected regions based on the proposed theoretically guaranteed importance function, which considers utility, representativeness, robustness, and synergy. Without requiring additional fine-tuning, HCS enables VLMs to achieve rapid understandings of unseen scenes at any scale using minimal interpretable regions while mitigating insufficient feature density. HCS is a plug-and-play method that is compatible with any VLM. Experiments demonstrate that HCS achieves superior performance and universality in various tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
