arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.09127cs.CLcs.AI
收藏

ERNIE-GeoL:一种地理语言预训练模型及其在百度地图中的应用

ERNIE-GeoL: A Geography-and-Language Pre-trained Model and its Applications in Baidu Maps

Baidu Inc.(百度公司)

Jizhou Huang, Haifeng Wang, Yibo Sun, Yunsheng Shi, Zhengjie Huang, An Zhuo, Shikun Feng

摘要:

预训练模型(PTM)已成为自然语言处理和计算机视觉下游任务的基本支柱。尽管通过将通用PTM应用于百度地图的地理相关任务获得了初步收益,但随着时间的推移,观察到了明显的性能平台。造成这一停滞的主要原因之一是缺乏通用PTM的现成地理知识。为了解决这个问题,在本文中,我们提出了ERNIE-GeoL,这是一个地理和语言预训练模型,旨在改善百度地图上与地理相关的任务。ERNIE-GeoL经过精心设计,通过对包含丰富地理知识的异构图生成的大规模数据进行预训练来学习地理语言的通用表示。在大规模真实数据集上进行的大量定量和定性实验证明了ERNIE-GeoL的优越性和有效性。ERNIE-GeoL已于2021年4月在百度地图的生产环境中部署,显著提升了下游各项任务的性能。这表明ERNIE-GeoL可以作为广泛的地理相关任务的基本骨干。

英文摘要:

Pre-trained models (PTMs) have become a fundamental backbone for downstream tasks in natural language processing and computer vision. Despite initial gains that were obtained by applying generic PTMs to geo-related tasks at Baidu Maps, a clear performance plateau over time was observed. One of the main reasons for this plateau is the lack of readily available geographic knowledge in generic PTMs. To address this problem, in this paper, we present ERNIE-GeoL, which is a geography-and-language pre-trained model designed and developed for improving the geo-related tasks at Baidu Maps. ERNIE-GeoL is elaborately designed to learn a universal representation of geography-language by pre-training on large-scale data generated from a heterogeneous graph that contains abundant geographic knowledge. Extensive quantitative and qualitative experiments conducted on large-scale real-world datasets demonstrate the superiority and effectiveness of ERNIE-GeoL. ERNIE-GeoL has already been deployed in production at Baidu Maps since April 2021, which significantly benefits the performance of various downstream tasks. This demonstrates that ERNIE-GeoL can serve as a fundamental backbone for a wide range of geo-related tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑