arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.14994cs.IRcs.MM
收藏

GeoWINE:基于地理位置的Wiki、图像、新闻和事件检索

GeoWINE: Geolocation based Wiki, Image,News and Event Retrieval

Golsa Tahmasebzadeh, Endri Kacupaj, Eric Müller-Budack, Sherzod Hakimov, Jens Lehmann, Ralph Ewerth

摘要:

在社交媒体的背景下,对新闻或事件的地理位置推断已经成为一项非常重要的任务。在本文中,我们提出了GeoWINE(基于地理位置的维基图像新闻事件检索)演示,一个有效的模块化系统的多模态检索,预计只有一个单一的图像作为输入。GeoWINE系统由五个模块组成,以便从各种来源检索相关信息。第一个模块是用于图像的地理位置估计的最先进的模型。第二个模块使用维基数据知识图谱执行基于地理空间的查询以进行实体检索。第三个模块利用四种不同的图像嵌入表示,这是用来检索最相似的实体相比,输入图像。嵌入来自于地理位置估计、地点识别、基于ImageNet的图像分类以及它们的组合。最后两个模块执行从EventRegistry和知识图谱(OEKG)检索新闻和事件。GeoWINE为最终用户提供了一个直观的界面,并为专家提供了深刻的见解,以重新配置到个人设置。GeoWINE在Google Landmarks数据集上的图像实体标签预测方面取得了可喜的成果。该演示程序可在http://cleopatra.ijs.si/geowine/上公开获取。

英文摘要:

In the context of social media, geolocation inference on news or events has become a very important task. In this paper, we present the GeoWINE (Geolocation-based Wiki-Image-News-Event retrieval) demonstrator, an effective modular system for multimodal retrieval which expects only a single image as input. The GeoWINE system consists of five modules in order to retrieve related information from various sources. The first module is a state-of-the-art model for geolocation estimation of images. The second module performs a geospatial-based query for entity retrieval using the Wikidata knowledge graph. The third module exploits four different image embedding representations, which are used to retrieve most similar entities compared to the input image. The embeddings are derived from the tasks of geolocation estimation, place recognition, ImageNet-based image classification, and their combination. The last two modules perform news and event retrieval from EventRegistry and the Open Event Knowledge Graph (OEKG). GeoWINE provides an intuitive interface for end-users and is insightful for experts for reconfiguration to individual setups. The GeoWINE achieves promising results in entity label prediction for images on Google Landmarks dataset. The demonstrator is publicly available at http://cleopatra.ijs.si/geowine/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑