arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.09025cs.CVcs.AI
收藏

基于技能的视觉地理定位用于视觉-语言模型

Skill-Conditioned Visual Geolocation for Vision-Language Models

Southwest Jiaotong University(西南交通大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Zhejiang University(浙江大学)

Chenjie Yang, Yutian Jiang, Yutong Deng, Chenyu Wu

AI总结:

本文提出GeoSkill框架,通过技能图进化提升视觉语言模型的地理定位能力与自我进化能力,实验显示其在GeoRC任务中表现优异,且在外部数据集上具有良好的泛化能力。

摘要:

视觉-语言模型(VLMs)在图像地理定位任务中展现出潜力,但缺乏结构化的地理推理能力和自主进化能力。现有方法主要依赖隐式参数化记忆,常利用过时知识并产生幻觉推理。此外,当前推理过程是“一次性”的,缺乏基于推理结果的反馈循环。为此,我们提出GeoSkill,一种基于进化技能图的无训练框架。首先通过优化人类专家轨迹生成原子自然语言技能。执行时,GeoSkill利用推理模型进行直接推理,受当前技能图引导。持续增长方面,自主进化机制利用大模型对来自网络级数据的图像坐标对进行多轮推理。通过分析成功和失败轨迹,机制迭代合成和剪枝技能,有效扩展技能图并纠正地理偏差,无需参数更新。实验表明,GeoSkill在GeoRC任务中实现了地理定位准确性和推理可信度的优异表现,并在多样化外部数据集上保持了优越的泛化能力。此外,自主进化促进了新技能的出现,显著提升了系统对现实地理知识的认知。

英文摘要:

Vision-language models (VLMs) have shown a promising ability in image geolocation, but they still lack structured geographic reasoning and the capacity for autonomous self-evolution. Existing methods predominantly rely on implicit parametric memory, which often exploits outdated knowledge and generates hallucinated reasoning. Furthermore, current inference is a "one-off" process, lacking the feedback loops necessary for self-evolution based on reasoning outcomes. To address these issues, we propose GeoSkill, a training-free framework based on an evolving Skill-Graph. We first initialize the graph by refining human expert trajectories into atomic, natural-language skills. For execution, GeoSkill employs an inference model to perform direct reasoning guided by the current Skill-Graph. For continuous growth, an Autonomous Evolution mechanism leverages a larger model to conduct multiple reasoning rollouts on image-coordinate pairs sourced from web-scale data and verified real-world reasoning. By analyzing both successful and failed trajectories from these rollouts, the mechanism iteratively synthesizes and prunes skills, effectively expanding the Skill-Graph and correcting geographic biases without any parameter updates. Experiments demonstrate that GeoSkill achieves promising performance in both geolocation accuracy and reasoning faithfulness on GeoRC, while maintaining superior generalization across diverse external datasets. Furthermore, our autonomous evolution fosters the emergence of novel, verifiable skills, significantly enhancing the system's cognition of real-world geographic knowledge beyond isolated case studies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑