迈向全局最优的视觉上下文学习提示选择
Towards Global Optimal Visual In-Context Learning Prompt Selection
AI总结：
针对视觉上下文学习的提示选择问题，提出Partial2Global框架，通过列表级排序器和一致性感知聚合器近似识别全局最优提示，在多个视觉任务上实现了新的最优性能。
摘要：
视觉上下文学习（VICL）是一种将视觉基础模型迁移到新任务的流行方式，它利用上下文示例中包含的上下文信息来增强查询样本的学习和预测。VICL中的基本问题是如何选择最佳提示以尽可能激活其能力，这等价于对候选集中每个候选者的上下文行为进行测试并选出最佳者的排序问题。为了利用更合适的排序指标并利用候选集中更全面的信息，我们提出了一种新颖的上下文示例选择框架，以近似识别全局最优提示，即为每个查询样本从所有候选者中选择表现最佳的上下文示例。我们的方法名为Partial2Global，采用基于transformer的列表级排序器在多个候选者间提供更全面的比较，并使用一致性感知排序聚合器生成全局一致的排序。Partial2Global的有效性通过在前景分割、单目标检测和图像着色上的实验得到验证，证明Partial2Global与其他方法相比选择了一致更好的上下文示例，从而确立了新的最优水平。
英文摘要：
Visual In-Context Learning (VICL) is a prevailing way to transfer visual foundation models to new tasks by leveraging contextual information contained in in-context examples to enhance learning and prediction of query sample. The fundamental problem in VICL is how to select the best prompt to activate its power as much as possible, which is equivalent to the ranking problem to test the in-context behavior of each candidate in the alternative set and select the best one. To utilize more appropriate ranking metric and leverage more comprehensive information among the alternative set, we propose a novel in-context example selection framework to approximately identify the global optimal prompt, i.e. choosing the best performing in-context examples from all alternatives for each query sample. Our method, dubbed Partial2Global, adopts a transformer-based list-wise ranker to provide a more comprehensive comparison within several alternatives, and a consistency-aware ranking aggregator to generate globally consistent ranking. The effectiveness of Partial2Global is validated through experiments on foreground segmentation, single object detection and image colorization, demonstrating that Partial2Global selects consistently better in-context examples compared with other methods, and thus establish the new state-of-the-arts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
