CALM:一种用于时间序列流中持续、自适应且由 LLM 介导的异常检测框架
CALM: A Framework for Continuous, Adaptive, and LLM-Mediated Anomaly Detection in Time-Series Streams
AI总结:
本文提出 CALM 框架,基于 Apache Beam 与 TimesFm,通过持续微调闭环和 LLM-as-a-Judge 语义筛选,使时间序列流异常检测适应概念漂移,并在 TSB-UAD 上提升多数数据集 ROC AUC。
摘要:
在非平稳时间序列流中检测异常是众多工业和科学领域中的一项关键但具有挑战性的任务。离线训练的传统模型在面对概念漂移(即数据的底层统计属性随时间变化)时,会出现显著的性能下降。本文介绍 CALM(Continuous, Adaptive, and LLM-Mediated,持续、自适应且由 LLM 介导),这是一种新颖的端到端实时异常检测框架,旨在应对这一挑战。CALM 构建于 Apache Beam 分布式处理框架之上,并利用 TimesFm 基础模型进行基于预测的异常检测。该框架的新颖性体现在两项核心贡献。首先,它实现了闭环、持续微调机制,使异常检测模型能够近乎实时地适应不断演化的数据模式。其次,它引入了 LLM-as-a-Judge 组件,即一个大语言模型,用于对检测到的异常提供语义化、情境感知的判断,以筛选高质量训练数据集,并判定某个异常代表的是瞬时噪声还是有意义的模式转变。我们在综合性 TSB-UAD 基准上评估 CALM。结果表明,与静态的预训练基础模型相比,持续微调后的模型在大多数数据集上提高了 ROC AUC 分数,验证了我们这种自适应、由 LLM 引导的方法在动态流环境中维持高性能异常检测的有效性。
英文摘要:
The detection of anomalies in non-stationary time-series streams is a critical but challenging task across numerous industrial and scientific domains. Traditional models, trained offline, suffer significant performance degradation when faced with concept drift, where the underlying statistical properties of the data change over time. This paper introduces CALM (Continuous, Adaptive, and LLM-Mediated), a novel, end-to-end framework for real-time anomaly detection designed to address this challenge. CALM is built on the Apache Beam distributed processing framework and leverages the TimesFm foundation model for forecasting-based anomaly detection. The framework's novelty lies in two core contributions. First, it implements a closed-loop, continuous fine-tuning mechanism that allows the anomaly detection model to adapt to evolving data patterns in near real-time. Second, it introduces an LLM-as-a-Judge component, a Large Language Model that provides semantic, context-aware judgments on detected anomalies to curate a high-quality training dataset, deciding whether an anomaly represents transient noise or a meaningful pattern shift. We evaluate CALM on the comprehensive TSB-UAD benchmark. Our results demonstrate that the continuously fine-tuned model improves the ROC AUC score in most datasets compared to the static, pre-trained base model, validating the efficacy of our adaptive, LLM-guided approach to maintaining high-performance anomaly detection in dynamic streaming environments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。