发表机构
Université Côte d’Azur; Inria; CNRS（蔚蓝海岸大学; 法国国家信息与自动化研究所; 法国国家科学研究中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出利用专家指定的多格式属性微调特征提取器并学习分类头，结合自动筛选误分类数据供专家修正，在三个数据集上提升图像分类性能。
AI 中文摘要
多年来，人们对将领域专家知识整合到图像分类模型中寄予了厚望。已经探索了几种方法，概念瓶颈模型（CBMs）开辟了一条新的研究方向，催生了许多变体，最近又出现了基于概念的嵌入模型（CEMs）。CBM对每个概念采用二元编码，而CEM通过两个向量嵌入每个概念来扩展这一思想。然而，在现实场景中，领域专家的知识通常以由各种属性确定的概念来组织，每个属性以数值、数值范围、二元值或分类值进行编码。在这项工作中，我们首先微调一个图像特征提取器，用于对代表下游目标类别的属性进行分类，其中类别属性已由专家以各种编码格式指定。然后，从这些不同的属性中学习一个分类头来对目标对象进行分类。我们通过实验表明，这在三个数据集上提高了分类性能：Kaggle鱼类数据集、AWA2和一个更具挑战性的新木炭数据集。接着，我们提出了一种自动选择潜在错误分类数据的方法。在第二步中，专家被要求针对这些数据最终修改预测的属性，以改进分类。
英文摘要
A lot of expectations have been put for years on integrating domain expert knowledge in image classification models. Several approaches have been explored, Concept Bottleneck Models (CBMs) opened up a new avenue of research leading to many variants, and more recently to Concept-based Embedding Models (CEMs). CBM consider binary encoding of each concept, while CEM expands this idea by embedding each concept through two vectors. However in real-life scenarii, domain experts' knowledge is usually organized in concepts determined by various attributes, each attribute encoded either with numerical values, or range of values, or binary values, or categorical values. In this work, we first finetune an image feature extractor for classifying attributes representing the downstream object classes, where the class attributes have been specified by experts under various encoding formats. A classification head is then learnt from these various attributes to categorize target objects. We experimentally show that it improves the classification for three datasets: Kaggle fish dataset, AWA2 and a more challenging new wood charcoal dataset. We then propose an automatic selection of potential missclassified data. In this second step, experts are asked for those data to eventually modify the predicted attributes in order to improve the classification.
Comments28 pages, 14 figures
