渐进式时间窗加宽
Progressive Temporal Window Widening
摘要:
本文介绍了一种对批处理时间具有鲁棒性的数据流处理方案。流框架以固定的时间间隔批量处理检索到的流。在共同的设置中,模式识别算法独立地应用于每个批次。选择正确的时间间隔是困难的-一个模式可能不适合一个太短的时间间隔,但如果时间间隔太长,检测将被延迟,内存可能会耗尽。我们在这里提出渐进式窗口加宽,一种算法,用于逐渐增加间隔,以便以任何速度捕获模式,而不会出现不必要的延迟或内存溢出。该算法与计算机安全、系统监控、用户行为跟踪以及必须在线识别数据流中未知或持续时间变化的模式的其他应用相关。现代数据流处理框架普遍用于处理大量数据,而渐进式窗口加宽促进的自适应内存和CPU分配对其性能至关重要。
英文摘要:
This paper introduces a scheme for data stream processing which is robust to batch duration. Streaming frameworks process streams in batches retrieved at fixed time intervals. In a common setting a pattern recognition algorithm is applied independently to each batch. Choosing the right time interval is tough --- a pattern may not fit in an interval which is too short, but detection will be delayed and memory may be exhausted if the interval is too long. We propose here Progressive Window Widening, an algorithm for increasing the interval gradually so that patterns are caught at any pace without unnecessary delays or memory overflow. This algorithm is relevant to computer security, system monitoring, user behavior tracking, and other applications where patterns of unknown or varying duration must be recognized online in data streams. Modern data stream processing frameworks are ubiquitously used to process high volumes of data, and adaptive memory and CPU allocation, facilitated by Progressive Window Widening, is crucial for their performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。