从语言模型嵌入中恢复时间与地理信号
Recovering Temporal and Geographic Signals from Language Model Embeddings
- Universidad de Buenos Aires（布宜诺斯艾利斯大学）
- Instituto de Ciencias de la Computación, CONICET-UBA（计算机科学研究所，CONICET-UBA）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究提出一种基于投影的黑盒方法，仅利用输出嵌入即可从语言模型中恢复时间与地理信号，验证了嵌入编码时空信息，并为下游检索任务提供实用工具。
AI中文摘要：
理解语言模型嵌入是否编码结构化的现实世界信息，对于表示分析和信息检索都至关重要。我们使用一种简单的基于投影的方法，直接作用于输出嵌入，来研究时间与地理信号。给定一小部分种子示例，该方法在嵌入空间中定义一个轴，并根据文本或实体在该轴上的投影对其进行排序。我们的方法完全黑盒且与模型无关：它仅需要嵌入，无需访问模型权重、内部激活、辅助探针或额外训练。这使得它适用于仅通过API可用的现代嵌入模型，并提供了一种轻量级的方式来分析其表示空间中是否存在时间与空间维度。我们将该方法应用于时间和地理数据集，发现嵌入投影能够恢复有意义的时序和空间结构。这些结果提供了证据表明输出嵌入编码了与时间和空间相关的信号，同时也为可解释性以及下游的时间和地理信息检索任务（如时间排序、地理排名和标注）提供了实用工具。
英文摘要：
Understanding whether language-model embeddings encode structured real-world information is important for both representation analysis and information retrieval. We study this question for temporal and geographic signals using a simple projection-based method that operates directly on output embeddings. Given a small set of seed examples, the method defines an axis in embedding space and ranks texts or entities by their projection onto that axis. Our approach is fully black-box and model-agnostic: it requires only embeddings, without access to model weights, internal activations, auxiliary probes, or additional training. This makes it applicable to modern embedding models available only through APIs and provides a lightweight way to analyze whether temporal and spatial dimensions are present in their representation spaces. We apply the method to temporal and geographic datasets and find that embedding projections recover meaningful chronological and spatial structure. These results provide evidence that output embeddings encode signals relevant to time and space, while also offering a practical tool for interpretability and for downstream temporal and geographic information retrieval tasks, such as temporal ordering, geographic ranking, and tagging.
