GeoToken：基于下一个 token 预测的图像分层地理定位
GeoToken: Hierarchical Geolocalization of Images via Next Token Prediction
AI总结：
提出分层序列预测框架 GeoToken，利用 S2 网格将图像地理定位转化为自回归 token 预测任务，结合束搜索与多样本推理等测试时计算策略，在 Im2GPS3k 和 YFCC4k 数据集上实现最优性能。
摘要：
图像地理定位，即确定图像地理来源的任务，面临重大挑战，这主要由于不同地点间的视觉相似性以及巨大的搜索空间。为了解决这些问题，我们提出了一种分层序列预测方法，其灵感来源于人类将位置从广阔区域缩小至具体地址的方式。类似地，我们的模型分层预测地理 token，首先识别一个大致区域，然后逐步将预测细化至越来越精确的位置。我们的方法不依赖显式的语义划分，而是使用 S2 cells（一种嵌套的多分辨率全球网格），并在视觉输入和先前预测的条件下，顺序预测更精细级别的网格。该过程类似于大型语言模型中的自回归文本生成。与语言建模一样，最终性能不仅取决于训练，还取决于推理时策略。我们研究了用于自回归采样的多种自顶向下遍历方法，并融入了语言模型中使用的测试时计算缩放技术。具体而言，我们集成了束搜索和多样本推理，同时探索了各种选择策略以确定最终输出。这使模型能够通过探索层级结构中的多条合理路径来管理不确定性。我们在 Im2GPS3k 和 YFCC4k 数据集上评估了该方法，并与两组不同的基线进行对比：不使用多模态大型语言模型（MLLM）的基线以及使用 MLLM 的基线。在无 MLLM 的设置下，我们的模型在几乎所有指标上超越了其他可比基线，实现了最先进的性能，准确率提升高达 13.9%。在引入 MLLM 增强后，我们的模型优于所有基线，在所有指标上创下新的最先进纪录。源代码可在 https://github.com/NNargesNN/GeoToken 获取。
英文摘要：
Image geolocalization, the task of determining an image's geographic origin, poses significant challenges, largely due to visual similarities across disparate locations and the large search space. To address these issues, we propose a hierarchical sequence prediction approach inspired by how humans narrow down locations from broad regions to specific addresses. Analogously, our model predicts geographic tokens hierarchically, first identifying a general region and then sequentially refining predictions to increasingly precise locations. Rather than relying on explicit semantic partitions, our method uses S2 cells, a nested, multiresolution global grid, and sequentially predicts finer-level cells conditioned on visual inputs and previous predictions. This procedure mirrors autoregressive text generation in large language models. Much like in language modeling, final performance depends not only on training but also on inference-time strategy. We investigate multiple top-down traversal methods for autoregressive sampling, incorporating techniques from test-time compute scaling used in language models. Specifically, we integrate beam search and multi-sample inference while exploring various selection strategies to determine the final output. This enables the model to manage uncertainty by exploring multiple plausible paths through the hierarchy. We evaluate our method on the Im2GPS3k and YFCC4k datasets against two distinct sets of baselines: those that operate without a Multimodal Large Language Model (MLLM) and those that leverage one. In the MLLM-free setting, our model surpasses other comparable baselines on nearly all metrics, achieving state-of-the-art performance with accuracy gains of up to 13.9%. When augmented with an MLLM, our model outperforms all baselines, setting a new state-of-the-art across all metrics. The source code is available at https://github.com/NNargesNN/GeoToken.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
