arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.00813cs.CLcs.AIcs.CY
收藏

UrbanGPT:时空大语言模型

UrbanGPT: Spatio-Temporal Large Language Models

The University of Hong Kong(香港大学);South China University of Technology(华南理工大学);Baidu Inc.(百度公司)

Zhonghang Li, Lianghao Xia, Jiabin Tang, Yong Xu, Lei Shi, Long Xia, Dawei Yin, Chao Huang

AI总结:

针对城市时空预测中数据稀缺、现有模型泛化性不足的问题,提出融合时空依赖编码器与指令微调的UrbanGPT时空大语言模型,在多任务公开数据集上性能超越现有最优基线,为零样本时空学习提供了新方案。

摘要:

时空预测旨在预测并洞察城市环境在时间与空间维度上不断变化的动态特征,其目标是预判城市生活各方面(包括交通、人口流动和犯罪率)的未来模式、趋势与事件。尽管已有大量研究致力于开发神经网络技术以实现时空数据的精准预测,但值得注意的是,这类方法大多高度依赖充足的标注数据来生成精确的时空表征。遗憾的是,在实际城市感知场景中,数据稀缺问题普遍存在。因此,构建一种能在各类时空学习场景下具备强泛化能力的时空模型十分必要。受大语言模型(LLMs)卓越成就的启发,我们的目标是打造一种能在广泛的下游城市任务中展现出优异泛化能力的时空大语言模型。为实现这一目标,我们提出了UrbanGPT,该模型将时空依赖编码器与指令微调范式无缝融合。这种整合使得大语言模型能够理解时间与空间之间复杂的相互依赖关系,从而在数据稀缺的场景下实现更全面、更准确的预测。为验证我们方法的有效性,我们在多个涵盖不同时空预测任务的公开数据集上开展了大量实验。结果一致表明,我们精心设计架构的UrbanGPT始终优于当前最优的基线模型。这些发现凸显了构建面向时空学习的大语言模型的潜力,尤其是在标注数据稀缺的零样本场景下。

英文摘要:

Spatio-temporal prediction aims to forecast and gain insights into the ever-changing dynamics of urban environments across both time and space. Its purpose is to anticipate future patterns, trends, and events in diverse facets of urban life, including transportation, population movement, and crime rates. Although numerous efforts have been dedicated to developing neural network techniques for accurate predictions on spatio-temporal data, it is important to note that many of these methods heavily depend on having sufficient labeled data to generate precise spatio-temporal representations. Unfortunately, the issue of data scarcity is pervasive in practical urban sensing scenarios. Consequently, it becomes necessary to build a spatio-temporal model with strong generalization capabilities across diverse spatio-temporal learning scenarios. Taking inspiration from the remarkable achievements of large language models (LLMs), our objective is to create a spatio-temporal LLM that can exhibit exceptional generalization capabilities across a wide range of downstream urban tasks. To achieve this objective, we present the UrbanGPT, which seamlessly integrates a spatio-temporal dependency encoder with the instruction-tuning paradigm. This integration enables LLMs to comprehend the complex inter-dependencies across time and space, facilitating more comprehensive and accurate predictions under data scarcity. To validate the effectiveness of our approach, we conduct extensive experiments on various public datasets, covering different spatio-temporal prediction tasks. The results consistently demonstrate that our UrbanGPT, with its carefully designed architecture, consistently outperforms state-of-the-art baselines. These findings highlight the potential of building large language models for spatio-temporal learning, particularly in zero-shot scenarios where labeled data is scarce.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑