使用多模态嵌入的在线论坛交互式搜索和探索
Interactive Search and Exploration in Online Discussion Forums Using Multimodal Embeddings
摘要：
在本文中，我们提出了一种新的交互式多模态学习系统，便于搜索和探索大型网络的社会多媒体用户。它允许分析人员识别和选择感兴趣的用户，并在交互式学习环境中找到类似的用户。我们的方法是基于新的用户，单词和概念的多模态表示，我们同时通过部署一个通用的神经嵌入模型来学习。我们显示这些表示是有用的，不仅分类用户，而且自动生成用户和社区配置文件。受传统摘要方法的启发，我们通过从所有可用的模态（即文本，图像和用户模态）中选择多样化和代表性的内容来创建配置文件。使用人工演员，它模拟用户行为的相关反馈的情况下，该方法的有用性进行评估。为了评估我们的多模态表示的质量，比较不同的嵌入策略，并确定不同模态的重要性，进行了多个实验。我们展示了所提出的方法的能力，两个不同的多媒体集合，来自暴力的在线极端主义论坛Stormfront和微博平台Twitter，这是特别有趣的，由于高语义级别的讨论，他们的功能。
英文摘要：
In this paper we present a novel interactive multimodal learning system, which facilitates search and exploration in large networks of social multimedia users. It allows the analyst to identify and select users of interest, and to find similar users in an interactive learning setting. Our approach is based on novel multimodal representations of users, words and concepts, which we simultaneously learn by deploying a general-purpose neural embedding model. We show these representations to be useful not only for categorizing users, but also for automatically generating user and community profiles. Inspired by traditional summarization approaches, we create the profiles by selecting diverse and representative content from all available modalities, i.e. the text, image and user modality. The usefulness of the approach is evaluated using artificial actors, which simulate user behavior in a relevance feedback scenario. Multiple experiments were conducted in order to evaluate the quality of our multimodal representations, to compare different embedding strategies, and to determine the importance of different modalities. We demonstrate the capabilities of the proposed approach on two different multimedia collections originating from the violent online extremism forum Stormfront and the microblogging platform Twitter, which are particularly interesting due to the high semantic level of the discussions they feature.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
