利用层间变换平滑性进行分布外检测
Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness
AI总结:
针对现有OOD检测方法依赖训练数据或需干预训练的局限,提出基于Transformer层间变换平滑性的检测方法BLOOD,无需训练数据即可用于预训练模型,在文本分类任务上性能优于同资源需求的方法。
摘要:
有效的分布外(OOD)检测对于可靠的机器学习模型至关重要,但当前大多数方法因需要访问训练数据或干预训练等要求,实际应用受到限制。我们提出了一种基于网络中间层间变换平滑性的Transformer OOD数据检测新方法BLOOD,该方法适用于预训练模型,无需访问训练数据。BLOOD利用了分布内(ID)数据的层间表示变换比OOD数据的对应变换更平滑的趋势,我们也通过实验验证了这一特性。我们在多个文本分类任务上用Transformer网络评估了BLOOD,证明其性能优于资源需求相当的方法。我们的分析还表明,学习较简单任务时,OOD数据变换保持原有锐度,而任务更复杂时锐度会上升。
英文摘要:
Effective out-of-distribution (OOD) detection is crucial for reliable machine learning models, yet most current methods are limited in practical use due to requirements like access to training data or intervention in training. We present a novel method for detecting OOD data in Transformers based on transformation smoothness between intermediate layers of a network (BLOOD), which is applicable to pre-trained models without access to training data. BLOOD utilizes the tendency of between-layer representation transformations of in-distribution (ID) data to be smoother than the corresponding transformations of OOD data, a property that we also demonstrate empirically. We evaluate BLOOD on several text classification tasks with Transformer networks and demonstrate that it outperforms methods with comparable resource requirements. Our analysis also suggests that when learning simpler tasks, OOD data transformations maintain their original sharpness, whereas sharpness increases with more complex tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。