将地理多样性知识融入提示以提升目标识别中的地理鲁棒性
Incorporating Geo-Diverse Knowledge into Prompting for Increased Geographical Robustness in Object Recognition
AI总结：
本文提出将地理多样性知识融入提示，并设计地理知识正则化，使在源地理上训练的软提示能泛化到未见目标地理，从而提升CLIP目标识别的地理鲁棒性。
摘要：
现有目标识别模型已被证明在多样化地理场景中缺乏鲁棒性，原因在于设计和上下文方面的领域偏移。类别表示需要被调整，以更准确地反映这些偏移下的目标概念。在缺乏目标地理区域训练数据的情况下，我们假设类别的具有地理多样性的描述性知识可以增强鲁棒性。为此，我们探索了从大型语言模型中探测基于地理的目标知识的可行性，并研究了将知识整合到CLIP的零样本和可学习软提示中的效果。在这一探索中，我们提出了地理知识正则化，以确保在源地理集合上训练的软提示能够泛化到未见过的目标地理集合。仅在欧洲数据上训练时，在DollarStreet数据集上相比提示基线的准确率提升在来自非洲/亚洲/美洲的目标数据上最高达到+2.8/1.2/1.6，在最困难类别上总体提升+4.6。与少样本目标训练相比，我们展示了具有竞争力的性能，并提供了分析以指导未来关于地理鲁棒性的研究。
英文摘要：
Existing object recognition models have been shown to lack robustness in diverse geographical scenarios due to domain shifts in design and context. Class representations need to be adapted to more accurately reflect an object concept under these shifts. In the absence of training data from target geographies, we hypothesize that geographically diverse descriptive knowledge of categories can enhance robustness. For this purpose, we explore the feasibility of probing a large language model for geography-based object knowledge, and we examine the effects of integrating knowledge into zero-shot and learnable soft prompting with CLIP. Within this exploration, we propose geography knowledge regularization to ensure that soft prompts trained on a source set of geographies generalize to an unseen target set. Accuracy gains over prompting baselines on DollarStreet while training only on Europe data are up to +2.8/1.2/1.6 on target data from Africa/Asia/Americas, and +4.6 overall on the hardest classes. Competitive performance is shown vs. few-shot target training, and analysis is provided to direct future study of geographical robustness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
