arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.01611cs.CVcs.AI
收藏

Google 即所需:面向轻量多模态多任务分类模型的半监督迁移学习策略

Google is all you need: Semi-Supervised Transfer Learning Strategy For Light Multimodal Multi-Task Classification Model

The University of Sydney(悉尼大学)

Haixu Liu, Penghao Jiang, Zerui Tao

AI总结:

本文提出一种结合CNN图像识别与NLP文本分析的轻量多模态多标签分类模型,通过半监督迁移学习与融合模块提升标注准确性,实验验证其作为自动图像标注系统的有效性。

摘要:

随着数字图像数据量的增加,图像分类的有效性不断增强。本研究引入了一个鲁棒的多标签分类系统,旨在为单张图像分配多个标签,以应对图像可能与多个类别相关联的复杂性(类别数从1到19,不包括12)。我们提出了一种多模态分类器,将先进的图像识别算法与自然语言处理(NLP)模型相结合,并引入融合模块来整合这些不同的模态。整合文本数据的目的是通过提供仅靠视觉分析无法完全捕捉的上下文理解来提高标签预测的准确性。我们提出的分类模型将用于图像处理的卷积神经网络(CNN)与用于分析文本描述(即图像说明)的NLP技术相结合。该方法包括严格的训练和验证阶段,每个模型组件均通过消融实验进行验证和分析。初步结果证明了该分类器的准确性和效率,凸显了其作为自动图像标注系统的潜力。

英文摘要:

As the volume of digital image data increases, the effectiveness of image classification intensifies. This study introduces a robust multi-label classification system designed to assign multiple labels to a single image, addressing the complexity of images that may be associated with multiple categories (ranging from 1 to 19, excluding 12). We propose a multi-modal classifier that merges advanced image recognition algorithms with Natural Language Processing (NLP) models, incorporating a fusion module to integrate these distinct modalities. The purpose of integrating textual data is to enhance the accuracy of label prediction by providing contextual understanding that visual analysis alone cannot fully capture. Our proposed classification model combines Convolutional Neural Networks (CNN) for image processing with NLP techniques for analyzing textual description (i.e., captions). This approach includes rigorous training and validation phases, with each model component verified and analyzed through ablation experiments. Preliminary results demonstrate the classifier's accuracy and efficiency, highlighting its potential as an automatic image-labeling system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑