第五届GCAIAC零样本指代表达理解挑战赛解决方案
The Solution for the 5th GCAIAC Zero-shot Referring Expression Comprehension Challenge
AI总结：
针对零样本指代表达理解任务，结合视觉提示组合与文本提示影响，采用适配数据特征的联合预测方法，在第五届GCAIAC挑战赛A、B榜分别取得84.825、71.460的准确率，获得第一名。
摘要：
本报告提出了一种零样本指代表达理解任务的解决方案。近年来，视觉-语言多模态基础模型（如CLIP、SAM）作为主流研究的基石受到了广泛关注。多模态基础模型的关键应用之一在于其能够泛化到零样本下游任务。与传统指代表达理解不同，零样本指代表达理解旨在将预训练视觉-语言模型直接应用于任务，无需进行特定训练。近期研究通过引入视觉提示，提升了多模态基础模型在指代表达理解任务中的零样本性能。为应对零样本指代表达理解挑战赛，我们引入了视觉提示组合，同时考虑文本提示的影响，采用适配数据特征的联合预测方法。最终，我们的方法在A榜单上取得了84.825的准确率，在B榜单上取得了71.460的准确率，位列第一名。
英文摘要：
This report presents a solution for the zero-shot referring expression comprehension task. Visual-language multimodal base models (such as CLIP, SAM) have gained significant attention in recent years as a cornerstone of mainstream research. One of the key applications of multimodal base models lies in their ability to generalize to zero-shot downstream tasks. Unlike traditional referring expression comprehension, zero-shot referring expression comprehension aims to apply pre-trained visual-language models directly to the task without specific training. Recent studies have enhanced the zero-shot performance of multimodal base models in referring expression comprehension tasks by introducing visual prompts. To address the zero-shot referring expression comprehension challenge, we introduced a combination of visual prompts and considered the influence of textual prompts, employing joint prediction tailored to the data characteristics. Ultimately, our approach achieved accuracy rates of 84.825 on the A leaderboard and 71.460 on the B leaderboard, securing the first position.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
