预训练Transformer揭示人类移动数据中的有意义模式
Pre-trained Transformer Uncovers Meaningful Patterns in Human Mobility Data
AI总结：
本研究通过在国家级未标记人类移动数据上预训练Transformer，得到的嵌入经微调可理解地理与移动模式，在多类相关任务中最高提升38%性能，为下游应用提供稳健表征。
摘要：
我们通过实证证明，在国家级未标记人类移动数据上预训练的Transformer所学习的嵌入向量，经微调后能够深入理解目标地理区域及其对应的移动模式。借助一个适配框架，我们评估了预训练嵌入在封装与人类移动直接或间接相关的广泛概念方面的性能，涵盖地理位置、距离等基础概念，还延伸至行政区划、土地覆盖等更复杂的结构。大量实证分析显示，预训练带来了显著的性能提升，在树木覆盖回归等任务中提升幅度最高达38%。我们将这一结果归因于预训练能够挖掘原始数据中隐藏的有意义模式，有助于对相关高层概念进行建模。这些预训练嵌入成为区域和轨迹的稳健表征，对广泛的下游应用具有潜在价值。
英文摘要：
We empirically demonstrate that a transformer pre-trained on country-scale unlabeled human mobility data learns embeddings capable, through fine-tuning, of developing a deep understanding of the target geography and its corresponding mobility patterns. Utilizing an adaptation framework, we evaluate the performance of our pre-trained embeddings in encapsulating a broad spectrum of concepts directly and indirectly related to human mobility. This includes basic notions, such as geographic location and distance, and extends to more complex constructs, such as administrative divisions and land cover. Our extensive empirical analysis reveals a substantial performance boost gained from pre-training, reaching up to 38% in tasks such as tree-cover regression. We attribute this result to the ability of the pre-training to uncover meaningful patterns hidden in the raw data, beneficial for modeling relevant high-level concepts. The pre-trained embeddings emerge as robust representations of regions and trajectories, potentially valuable for a wide range of downstream applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
