发表机构
University of Ljubljana; Faculty of Electrical Engineering; Faculty of Computer and Information Science（卢布尔雅那大学; 电气工程学院; 计算机与信息科学学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
SCOUT是一个端到端框架，可通过机械可解释性发现人脸识别模板中的语义概念，实现可控的身份感知模板编辑，且对身份匹配影响极小。
AI 中文摘要
人脸识别模板是紧凑的身份表示，同时也编码了关于面部外观的丰富语义信息。已有研究表明，模板可以被逆转为图像，或通过图像编辑流水线进行间接操作，但在模板空间中进行直接语义编辑在很大程度上仍未被探索。现有人脸识别可解释性方法通常依赖人工神经元检查或预定义的属性标签，这限制了可扩展性和语义灵活性。为解决这一差距，我们提出SCOUT（Semantic Concept Discovery for Open-VocabUlary Editing of Face Recognition Templates），这是一个使用机械可解释性来发现和直接操作人脸识别模板中语义概念的端到端框架。SCOUT学习稀疏模板表示，从自然语言描述中为潜在特征生成语义假设，并验证其稳定性。得到的特征充当可控的语义方向以进行直接编辑，避免了成本高昂的编辑-重新编码流水线。对使用CNN、ViT和Swin骨干网络的人脸识别模型进行的实验表明，SCOUT发现了超出标准属性标签的可解释概念，并实现了可控的、感知身份的模板操作，对身份匹配的影响可忽略不计。我们进一步表明，编辑后的模板随后可以通过独立的逆模型进行解码以用于可视化和评估。
英文摘要
Face recognition templates are compact identity representations, yet they also encode rich semantic information about facial appearance. Prior work has shown that templates can be inverted to images or indirectly manipulated through image-editing pipelines, but direct semantic editing in template space remains largely unexplored. Existing interpretability methods for face recognition often rely on manual neuron inspection or predefined attribute labels, limiting scalability and semantic flexibility. To address this gap, we propose SCOUT (Semantic Concept Discovery for Open-VocabUlary Editing of Face Recognition Templates), an end-to-end framework for discovering and directly manipulating semantic concepts in face recognition templates using mechanistic interpretability. SCOUT learns sparse template representations, generates semantic hypotheses for latent features from natural-language descriptions, and validates their stability. The resulting features act as controllable semantic directions for direct editing, avoiding costly edit--re-encode pipelines. Experiments with face recognition models using CNN, ViT, and Swin backbones show that SCOUT discovers interpretable concepts beyond standard attribute labels and enables controllable, identity-aware template manipulation with negligible impact on identity matching. We further show that edited templates can subsequently be decoded with independent inversion models for visualization and evaluation.
