arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1801.10253cs.CLcs.IRcs.MM
收藏

新模态:预测、预期和检索中的挑战

The New Modality: Emoji Challenges in Prediction, Anticipation, and Retrieval

Spencer Cappallo, Stacey Svetlichnaya, Pierre Garrigues, Thomas Mensink, Cees G. M. Snoek

摘要:

在过去的十年里,表情符号已经成为一种新的和广泛的数字通信形式,跨越了不同的社交网络和口语。我们建议把这些表意文字作为一种新的模态,在他们自己的权利,在他们的语义结构不同的文本中,他们往往是嵌入以及他们类似的图像。表情符号作为一种新的表现形式,为表情符号的表现和互动提供了新的可能性。在本文中,我们探讨了自然产生的挑战,通过多媒体研究的镜头考虑表情符号的模态。具体来说,emoji可以与其他常见形式(如文本和图像)相关的方式。为此,我们首先展示了从Twitter收集的真实世界emoji使用情况的大规模数据集。该数据集包含文本-表情符号和图像-表情符号关系的示例。我们目前的基线结果的挑战预测表情符号从文本和图像,使用国家的最先进的神经网络。此外,我们首先考虑如何解释新的、看不见的表情符号的问题--这是一个相关的问题,因为表情符号词汇每年都在不断扩大。最后,我们提出了使用表情符号作为查询的多媒体检索结果。

英文摘要:

Over the past decade, emoji have emerged as a new and widespread form of digital communication, spanning diverse social networks and spoken languages. We propose to treat these ideograms as a new modality in their own right, distinct in their semantic structure from both the text in which they are often embedded as well as the images which they resemble. As a new modality, emoji present rich novel possibilities for representation and interaction. In this paper, we explore the challenges that arise naturally from considering the emoji modality through the lens of multimedia research. Specifically, the ways in which emoji can be related to other common modalities such as text and images. To do so, we first present a large scale dataset of real-world emoji usage collected from Twitter. This dataset contains examples of both text-emoji and image-emoji relationships. We present baseline results on the challenge of predicting emoji from both text and images, using state-of-the-art neural networks. Further, we offer a first consideration into the problem of how to account for new, unseen emoji - a relevant issue as the emoji vocabulary continues to expand on a yearly basis. Finally, we present results for multimedia retrieval using emoji as queries.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑