METEOR：从多模态数据流中学习记忆和时间有效表示
METEOR: Learning Memory and Time Efficient Representations from Multi-modal Data Streams
摘要：
许多学习任务涉及多模态数据流，其中来自不同模式的连续数据传达关于对象的全面描述。在这种情况下，一个主要的挑战是如何有效地解释复杂环境中的多模态信息。这激发了许多关于从多模态数据流中学习无监督表示的研究。这些研究旨在了解更高层次的上下文信息（例如，Twitter消息）通过联合学习不同模态中的较低级别语义单元的嵌入（例如，Twitter消息的文本、用户和位置）。然而，这些方法直接将每个低级语义单元与连续的嵌入向量相关联，这导致高的存储需求。因此，在低存储器设备（例如，移动设备）成为问题。为了解决这个问题，我们提出了METEOR，一种新的记忆和时间有效的在线表示学习技术，它：（1）通过在语义上有意义的组内共享参数来学习多模态数据的紧凑表示，并保持领域无关的语义;（2）可以使用并行处理来加速，以适应不同的流速率，同时捕获单元的时间变化;以及（3）可以容易地扩展以捕获与多模态数据流相关的隐式/显式外部知识。我们使用两种类型的多模态数据流（即，社交媒体流和购物交易流）以展示其适应不同领域的能力。我们的研究结果表明，METEOR保留了表示的质量，同时减少了约80%的内存使用量相比，传统的内存密集型嵌入。
英文摘要：
Many learning tasks involve multi-modal data streams, where continuous data from different modes convey a comprehensive description about objects. A major challenge in this context is how to efficiently interpret multi-modal information in complex environments. This has motivated numerous studies on learning unsupervised representations from multi-modal data streams. These studies aim to understand higher-level contextual information (e.g., a Twitter message) by jointly learning embeddings for the lower-level semantic units in different modalities (e.g., text, user, and location of a Twitter message). However, these methods directly associate each low-level semantic unit with a continuous embedding vector, which results in high memory requirements. Hence, deploying and continuously learning such models in low-memory devices (e.g., mobile devices) becomes a problem. To address this problem, we present METEOR, a novel MEmory and Time Efficient Online Representation learning technique, which: (1) learns compact representations for multi-modal data by sharing parameters within semantically meaningful groups and preserves the domain-agnostic semantics; (2) can be accelerated using parallel processes to accommodate different stream rates while capturing the temporal changes of the units; and (3) can be easily extended to capture implicit/explicit external knowledge related to multi-modal data streams. We evaluate METEOR using two types of multi-modal data streams (i.e., social media streams and shopping transaction streams) to demonstrate its ability to adapt to different domains. Our results show that METEOR preserves the quality of the representations while reducing memory usage by around 80% compared to the conventional memory-intensive embeddings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
