arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.09858cs.LGcs.CV
收藏

用于分布外检测的极简激活整形方法

Extremely Simple Activation Shaping for Out-of-Distribution Detection

ML Collective(ML集体);Faculty of Technical Sciences, University of Novi Sad(诺威萨大学技术科学学院);Google Research, Brain Team(谷歌研究院大脑团队)

Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, Rosanne Liu

AI总结:

针对现有分布外检测需额外训练或网络修改的问题,提出极简事后激活整形方法ASH,推理时移除深层大部分激活值,无需训练统计量,在ImageNet上实现最优OOD检测且不明显降低分布内准确率。

摘要:

机器学习模型训练与部署的分离意味着,部署阶段遇到的所有场景无法都在训练时被预见,因此仅依赖训练环节的改进存在局限性。分布外(OOD)检测是压力测试模型处理未知场景能力的重要领域:模型是否清楚自己何时“不知道”?现有OOD检测方法要么需要额外训练步骤、额外数据,要么要对已训练网络进行大幅修改。相比之下,本研究提出了一种极简的、事后(post-hoc)即时生效的激活整形方法ASH:将样本在网络深层的激活值移除很大比例(例如90%),对剩余部分(例如10%)进行简化或轻微调整。该整形操作在推理阶段实施,无需任何从训练数据中计算得到的统计量。实验表明,这种简单处理增强了分布内与分布外样本的区分度,可在ImageNet上实现当前最优的OOD检测性能,且不会明显降低分布内准确率。相关视频、动画和代码可访问:https://andrijazz.github.io/ash

英文摘要:

The separation between training and deployment of machine learning models implies that not all scenarios encountered in deployment can be anticipated during training, and therefore relying solely on advancements in training has its limits. Out-of-distribution (OOD) detection is an important area that stress-tests a model's ability to handle unseen situations: Do models know when they don't know? Existing OOD detection methods either incur extra training steps, additional data or make nontrivial modifications to the trained network. In contrast, in this work, we propose an extremely simple, post-hoc, on-the-fly activation shaping method, ASH, where a large portion (e.g. 90%) of a sample's activation at a late layer is removed, and the rest (e.g. 10%) simplified or lightly adjusted. The shaping is applied at inference time, and does not require any statistics calculated from training data. Experiments show that such a simple treatment enhances in-distribution and out-of-distribution distinction so as to allow state-of-the-art OOD detection on ImageNet, and does not noticeably deteriorate the in-distribution accuracy. Video, animation and code can be found at: https://andrijazz.github.io/ash

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑