发表机构
AImotion Bavaria, Technische Hochschule Ingolstadt（巴伐利亚人工智能运动中心，英戈尔施塔特应用科学大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出一种轻量、与模型无关的方法，通过辅助嵌入为预训练语言模型注入地理空间感知，在提升地理空间对齐性能的同时保持原有NLP任务表现，且计算高效、可泛化至多种模型。
AI 中文摘要
基于Transformer的预训练语言模型在广泛的NLP任务中表现出色，但在编码地理空间语义方面仍存在局限，导致对地名和空间实体的表征效果欠佳。本研究提出一种轻量、与模型无关的方法，用于向预训练嵌入中注入地理空间感知能力，无需修改分词器或进行代价高昂的重新训练。该方法通过将地名与其对应的经纬度相结合，用结构化地理信号增强输入表征，并采用聚焦位置的掩码策略，使文本表征更好地与现实世界的空间关系对齐。这种设计让模型在保留现有语义和句法知识的同时，融入地理空间上下文。实验结果表明，该方法在地理空间对齐方面取得显著提升，且在GLUE等标准NLP基准上保持相当的性能；其计算效率高，仅需数分钟的额外训练，且可在多种模型架构和规模上泛化。
英文摘要
Pretrained transformer-based language models achieve strong performance across a wide range of NLP tasks but remain limited in encoding geo-locational semantics, leading to suboptimal representations of place names and spatial entities. In this work, we propose a lightweight, model-agnostic approach for injecting geo-spatial awareness into pretrained embeddings without modifying the tokenizer or requiring costly retraining. Our method augments input representations with structured geographic signals by combining location names with their corresponding latitude and longitude, and employs a location-focused masking to better align textual representations with real-world spatial relationships. This design allows the model to incorporate geo-spatial context while preserving existing semantic and syntactic knowledge. Experimental results demonstrate substantial improvements in geo-spatial alignment while maintaining comparable performance on standard NLP benchmarks such as GLUE. The method is computationally efficient, requiring only minutes of additional training, and generalizes across multiple model architectures and scales.
CommentsAccepted for publication at the 29th International Conference on Text, Speech and Dialogue (TSD 2026)
DOI：10.1007/978-3-032-37249-9_14
