arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.08761cs.CVcs.LG

“鹰眼狐耳”:面向广义零样本学习的部件原型网络

`Eyes of a Hawk and Ears of a Fox': Part Prototype Network for Generalized Zero-Shot Learning

Arizona State University(亚利桑那州立大学);Lawrence Livermore National Lab(劳伦斯利弗莫尔国家实验室)

Joshua Feinglass, Jayaraman J. Thiagarajan, Rushil Anirudh, T. S. Jayram, Yezhou Yang

AI总结:

针对现有广义零样本学习方法仅用全局属性向量的局限,提出基于VINVL检测器的部件原型网络,通过区域属性注意力构建类别部件原型,在主流基准上表现更优。

摘要:

当前广义零样本学习(GZSL)方法均基于基础模型构建,这类模型仅考虑整张图像的单一类别属性向量表示。这是对新类别识别过程的过度简化——在新类别识别中,图像的不同区域可能具备来自不同已见类别的特性,因此拥有不同的主导属性。基于此,我们采用一种截然不同的方法:使用对属性信息敏感的预训练视觉-语言检测器VINVL(视觉-语言区域特征提取器)高效获取区域特征。通过一个可学习函数将区域特征映射为区域特定的属性注意力,用于构建类别部件原型。我们在由CUB、SUN和AWA2数据集组成的主流GZSL基准上开展实验,所提出的Part Prototype Network(PPN,部件原型网络)与其他主流基础模型相比取得了颇具竞争力的结果。相应的消融实验与分析表明,该方法实用性强,在可获取局部候选框时,相比全局属性注意力具有显著优势。

英文摘要:

Current approaches in Generalized Zero-Shot Learning (GZSL) are built upon base models which consider only a single class attribute vector representation over the entire image. This is an oversimplification of the process of novel category recognition, where different regions of the image may have properties from different seen classes and thus have different predominant attributes. With this in mind, we take a fundamentally different approach: a pre-trained Vision-Language detector (VINVL) sensitive to attribute information is employed to efficiently obtain region features. A learned function maps the region features to region-specific attribute attention used to construct class part prototypes. We conduct experiments on a popular GZSL benchmark consisting of the CUB, SUN, and AWA2 datasets where our proposed Part Prototype Network (PPN) achieves promising results when compared with other popular base models. Corresponding ablation studies and analysis show that our approach is highly practical and has a distinct advantage over global attribute attention when localized proposals are available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑