arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.06045cs.CV
收藏

OpenCIL:类增量学习中的分布外检测基准测试

OpenCIL: Benchmarking Out-of-Distribution Detection in Class-Incremental Learning

School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);School of Computing and Information Systems, Singapore Management University(新加坡管理大学计算与信息系统学院);Jiangxi Research Institute, Beihang University(北京航空航天大学江西研究院)

Wenjun Miao, Guansong Pang, Trong-Tung Nguyen, Ruohang Fang, Jin Zheng, Xiao Bai

AI总结:

针对类增量学习(CIL)场景下缺乏分布外(OOD)检测基准的问题,研究构建了OpenCIL基准,集成多类模型与检测方法形成基线,发现开放环境下CIL模型存在偏向性,并提出双向能量正则化方法缓解该问题。

摘要:

类增量学习(CIL)旨在构建一个既能逐步适配新类别,又能保留已学旧类别知识的模型。CIL场景下的分布外(OOD)检测,是指在保留这种增量学习能力的同时,能够拒识来自与已学类别不同分布的未知样本。这一能力对于在开放世界中部署CIL模型的安全性至关重要。然而,尽管CIL和OOD检测各自领域都取得了显著进展,目前仍缺乏一个系统性的大规模基准,来评估先进CIL模型检测OOD样本的能力。为填补这一空白,本研究开展了全面的实证研究,构建了名为$\textbf{OpenCIL}$的基准。为此,我们提出了两个原则性框架,将15种多样化的OOD检测方法集成到4种代表性CIL模型中,最终得到60个用于CIL场景下OOD检测的基线模型。实证评估在两个主流CIL数据集以及6个常用OOD数据集上开展。通过全面评估,我们得到一项关键发现:当CIL模型处于开放环境中时,会对OOD样本和新增类别产生严重的偏向性。受此启发,我们进一步提出了一种适用于CIL场景下OOD检测的新基线方法——双向能量正则化($\textbf{BER}$),该方法专门针对不同CIL模型设计,通过对旧类别和新类别同时施加能量正则化来缓解上述两种偏向。实验结果验证了该方法的优异性能。所有代码和数据集已在https://github.com/mala-lab/OpenCIL 开源。

英文摘要:

Class incremental learning (CIL) aims to learn a model that can not only incrementally accommodate new classes, but also maintain the learned knowledge of old classes. Out-of-distribution (OOD) detection in CIL is to retain this incremental learning ability, while being able to reject unknown samples that are drawn from different distributions of the learned classes. This capability is crucial to the safety of deploying CIL models in open worlds. However, despite remarkable advancements in the respective CIL and OOD detection, there lacks a systematic and large-scale benchmark to assess the capability of advanced CIL models in detecting OOD samples. To fill this gap, in this study we design a comprehensive empirical study to establish such a benchmark, named $\textbf{OpenCIL}$. To this end, we propose two principled frameworks for enabling four representative CIL models with 15 diverse OOD detection methods, resulting in 60 baseline models for OOD detection in CIL. The empirical evaluation is performed on two popular CIL datasets with six commonly-used OOD datasets. One key observation we find through our comprehensive evaluation is that the CIL models can be severely biased towards the OOD samples and newly added classes when they are exposed to open environments. Motivated by this, we further propose a new baseline for OOD detection in CIL, namely Bi-directional Energy Regularization ($\textbf{BER}$), which is specially designed to mitigate these two biases in different CIL models by having energy regularization on both old and new classes. Its superior performance is justified in our experiments. All codes and datasets are open-source at https://github.com/mala-lab/OpenCIL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑