arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.02803cs.CV
收藏

基于视觉Transformer的分布外数据快速可解释人脸识别

Fast and Interpretable Face Identification for Out-Of-Distribution Data Using Vision Transformers

Auburn University(奥本大学);Columbia University(哥伦比亚大学);Phenikaa University(Phenikaa大学);Carnegie Mellon University(卡内基梅隆大学)

Hai Phan, Cindy Le, Vu Le, Yihui He, Anh Totti Nguyen

AI总结:

针对现有分布外人脸识别方法速度慢的问题,提出基于交叉注意力的双图像ViT模型,准确率与DeepFace-EMD相当,推理速度超其两倍且具备可解释性。

摘要:

大多数人脸识别方法采用孪生神经网络在图像嵌入层面对比两张图像,但该技术易受遮挡(如戴口罩或太阳镜的人脸)及分布外数据影响。DeepFace-EMD(Phan等人,2022)通过先在图像层面、再在补丁层面对比两张图像,在分布外数据上取得了当前最优准确率,但其后续的补丁级重排序阶段因最优传输优化,具有高达$O(n^3 \log n)$的时间复杂度($n$为图像中的补丁数量)。本文提出一种新颖的双图像视觉Transformer(ViTs),通过交叉注意力在补丁层面对比两张图像。在CASIA Webface(Yi等人,2014)数据集的200万对图像上训练后,我们的模型在分布外数据上的准确率与DeepFace-EMD相当,但推理速度是DeepFace-EMD的两倍以上。此外,通过人类受试者研究,我们的模型借助交叉注意力可视化展现了良好的可解释性。我们相信本工作能启发更多将ViT用于人脸识别的探索。

英文摘要:

Most face identification approaches employ a Siamese neural network to compare two images at the image embedding level. Yet, this technique can be subject to occlusion (e.g. faces with masks or sunglasses) and out-of-distribution data. DeepFace-EMD (Phan et al. 2022) reaches state-of-the-art accuracy on out-of-distribution data by first comparing two images at the image level, and then at the patch level. Yet, its later patch-wise re-ranking stage admits a large $O(n^3 \log n)$ time complexity (for $n$ patches in an image) due to the optimal transport optimization. In this paper, we propose a novel, 2-image Vision Transformers (ViTs) that compares two images at the patch level using cross-attention. After training on 2M pairs of images on CASIA Webface (Yi et al. 2014), our model performs at a comparable accuracy as DeepFace-EMD on out-of-distribution data, yet at an inference speed more than twice as fast as DeepFace-EMD (Phan et al. 2022). In addition, via a human study, our model shows promising explainability through the visualization of cross-attention. We believe our work can inspire more explorations in using ViTs for face identification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑