arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.05608cs.CVcs.AIcs.CLcs.LG
收藏

可微异常检测实现鲁棒的深度多模态分析

Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis

University of Illinois at Chicago(伊利诺伊大学芝加哥分校)

Zhu Wang, Sourav Medya, Sathya N. Ravi

AI总结:

针对深度网络纯归纳式推理的局限及先验知识反向传播适配难题,提出融合显式知识图谱与可微OOD滤波层的端到端视觉-语言模型,在多下游任务上以更少样本和训练时间达到SOTA级性能。

摘要:

通常,深度网络模型在训练及对未见数据执行推理时是纯归纳式的。因此,众所周知,这类模型用于预测时往往无法捕捉群体层面上对象(或概念)间存在的语义信息与隐式依赖关系。此外,目前仍不清楚如何以适配反向传播的方式指定领域或先验模态知识,尤其是在大规模且含噪的场景中。在本研究中,我们提出了一种融合显式知识图谱的端到端视觉-语言模型。我们还引入了一个基于隐式网络算子的交互式分布外(OOD)层,该层用于过滤外部知识库带来的噪声。在实际应用中,我们在多个视觉与语言下游任务上测试了模型,包括不同数据集上的视觉问答、视觉推理以及图文检索。我们的实验表明,设计出性能与当前最优结果相当,但所需样本量和训练时间显著更少的模型是可行的。

英文摘要:

Often, deep network models are purely inductive during training and while performing inference on unseen data. Thus, when such models are used for predictions, it is well known that they often fail to capture the semantic information and implicit dependencies that exist among objects (or concepts) on a population level. Moreover, it is still unclear how domain or prior modal knowledge can be specified in a backpropagation friendly manner, especially in large-scale and noisy settings. In this work, we propose an end-to-end vision and language model incorporating explicit knowledge graphs. We also introduce an interactive out-of-distribution (OOD) layer using implicit network operator. The layer is used to filter noise that is brought by external knowledge base. In practice, we apply our model on several vision and language downstream tasks including visual question answering, visual reasoning, and image-text retrieval on different datasets. Our experiments show that it is possible to design models that perform similarly to state-of-art results but with significantly fewer samples and training time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑