AnoShift：一个用于无监督异常检测的分布偏移基准
AnoShift: A Distribution Shift Benchmark for Unsupervised Anomaly Detection
AI总结：
针对无监督学习缺乏分布偏移基准的问题，提出基于Kyoto-2006+的无监督异常检测基准AnoShift，通过划分IID、NEAR和FAR测试子集验证模型性能退化，并证明妥善处理分布偏移可使性能平均提升高达3%。
摘要：
分析数据的分布偏移是当今机器学习（ML）中不断发展的研究方向，促使了新兴基准的出现，这些基准专注于为研究ML模型的泛化特性提供合适的场景。现有基准主要集中在监督学习上，据我们所知，尚无针对无监督学习的基准。因此，我们引入了一个无监督异常检测基准，其数据随时间发生偏移，基于Kyoto-2006+构建，这是一个用于网络入侵检测的流量数据集。这类数据满足偏移输入分布的前提：它涵盖了较长的时间跨度（$10$年），并存在随时间自然发生的变化（例如用户修改其行为模式，以及软件更新）。我们首先使用基本的逐特征分析、t-SNE以及一种用于测量年份间整体分布距离的最优传输方法，突出了数据的非平稳性质。接着，我们提出了AnoShift，这是一种将数据划分为IID、NEAR和FAR测试子集的协议划分方法。我们使用从经典方法到深度学习的多种模型，验证了性能随时间的退化。最后，我们表明，通过认识分布偏移问题并妥善处理它，与假设独立同分布数据的经典训练相比，性能可以得到提升（平均而言，我们的方法提升了高达$3\%$）。数据集和代码可在 https://github.com/bit-ml/AnoShift/ 获取。
英文摘要：
Analyzing the distribution shift of data is a growing research direction in nowadays Machine Learning (ML), leading to emerging new benchmarks that focus on providing a suitable scenario for studying the generalization properties of ML models. The existing benchmarks are focused on supervised learning, and to the best of our knowledge, there is none for unsupervised learning. Therefore, we introduce an unsupervised anomaly detection benchmark with data that shifts over time, built over Kyoto-2006+, a traffic dataset for network intrusion detection. This type of data meets the premise of shifting the input distribution: it covers a large time span ($10$ years), with naturally occurring changes over time (eg users modifying their behavior patterns, and software updates). We first highlight the non-stationary nature of the data, using a basic per-feature analysis, t-SNE, and an Optimal Transport approach for measuring the overall distribution distances between years. Next, we propose AnoShift, a protocol splitting the data in IID, NEAR, and FAR testing splits. We validate the performance degradation over time with diverse models, ranging from classical approaches to deep learning. Finally, we show that by acknowledging the distribution shift problem and properly addressing it, the performance can be improved compared to the classical training which assumes independent and identically distributed data (on average, by up to $3\%$ for our approach). Dataset and code are available at https://github.com/bit-ml/AnoShift/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
