高效的大规模多模态分类
Efficient Large-Scale Multi-Modal Classification
摘要：
虽然早期的互联网主要是基于文本的，但现代数字世界正变得越来越多模式。在这里，我们研究多模态分类，其中一种模态是离散的，例如文本，另一种模态是连续的，例如从卷积神经网络传输的视觉表示。特别是，我们专注于我们必须能够快速分类大量数据的场景。我们研究了各种方法进行多模态融合，并分析其权衡分类精度和计算效率。我们的研究结果表明，包括连续信息提高了性能，在一系列的多模态分类任务的纯文本，即使是简单的融合方法。此外，我们还尝试将连续特征离散化，以进一步加快和简化融合过程。我们的研究结果表明，与离散化功能的融合优于纯文本分类，在一小部分的计算成本的完整的多模态融合，与改进的可解释性的额外好处。
英文摘要：
While the incipient internet was largely text-based, the modern digital world is becoming increasingly multi-modal. Here, we examine multi-modal classification where one modality is discrete, e.g. text, and the other is continuous, e.g. visual representations transferred from a convolutional neural network. In particular, we focus on scenarios where we have to be able to classify large quantities of data quickly. We investigate various methods for performing multi-modal fusion and analyze their trade-offs in terms of classification accuracy and computational efficiency. Our findings indicate that the inclusion of continuous information improves performance over text-only on a range of multi-modal classification tasks, even with simple fusion methods. In addition, we experiment with discretizing the continuous features in order to speed up and simplify the fusion process even further. Our results show that fusion with discretized features outperforms text-only classification, at a fraction of the computational cost of full multi-modal fusion, with the additional benefit of improved interpretability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
