首页/论文/1908.02858

arXiv 1908.028582019-08-07cs.LGcs.SYeess.SYstat.ML

HyperStream: a Workflow Engine for Streaming Data

Amazon（亚马逊）；University of Tartu（塔尔图大学）；University of Bristol（布里斯托大学）

Tom Diethe, Meelis Kull, Niall Twomey, Kacper Sokol, Hao Song, Miquel Perello-Nieto, Emma Tonkin, Peter Flach

英文摘要：

This paper describes HyperStream, a large-scale, flexible and robust software package, written in the Python language, for processing streaming data with workflow creation capabilities. HyperStream overcomes the limitations of other computational engines and provides high-level interfaces to execute complex nesting, fusion, and prediction both in online and offline forms in streaming environments. HyperStream is a general purpose tool that is well-suited for the design, development, and deployment of Machine Learning algorithms and predictive models in a wide space of sequential predictive problems. Source code, installation instructions, examples, and documentation can be found at: https://github.com/IRC-SPHERE/HyperStream.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

A Column Streaming-Based Convolution Engine and Mapping Algorithm for CNN-based Edge AI accelerators

Edge AI accelerators have been emerging as a solution for near customers' applications in areas such as unmanned aerial vehicles (UAVs), image recognition sensors, wearable devices, robotics, and remote sensing satellites. These applications not only require meeting performance targets but also meeting strict area and power constraints due to their portable mobility feature and limited power sources. As a result, a column streaming-based convolution engine has been proposed in this paper that includes column sets of processing elements design for flexibility in terms of the applicability for d…

2021-09-15cs.AIcs.LGcs.SY

基于内容的视频流媒体平台推荐引擎

Content-based Recommendation Engine for Video Streaming Platform

Department of Computer Science Tribhuvan University（计算机科学系 Tribhuvan 大学）

本文提出一种基于内容的视频推荐引擎，利用TF-IDF和余弦相似度实现个性化视频推荐，提升用户参与度和观看体验。

2025-12-29cs.IRcs.LG

注意力一次就足够：高效的流式推断与状态化Transformer

Attention Once Is All You Need: Efficient Streaming Inference with Stateful Transformers

LayerScale, Inc.（LayerScale公司）

本文提出基于状态化会话的数据驱动计算模型，通过增量式KV缓存降低预填充成本，实现查询延迟与上下文长度无关。Flash Queries利用空闲GPU周期预处理问题，提升流式推断效率，实测在市场数据基准上达到5.9倍速度提升。

2026-05-13cs.LG

AsyncFlow：一种用于高效LLM后训练的非同步流式强化学习框架

AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training

Huawei（华为）；Huawei Individual Researcher（华为独立研究者）；Individual Researcher（独立研究者）

提出非同步流式RL框架AsyncFlow，通过分布式存储与异步生产者-消费者工作流实现流水线重叠和动态负载均衡，解耦底层引擎，吞吐量提升1.59倍。

2026-09-14cs.AIcs.LG

A streaming feature-based compression method for data from instrumented infrastructure

Alan Turing Institute（艾伦·图灵研究所）；Imperial College London（伦敦帝国学院）；Autodesk Research（欧特克研究院）

An increasing amount of civil engineering applications are utilising data acquired from infrastructure instrumented with sensing devices. This data has an important role in monitoring the response of these structures to excitation, and evaluating structural health. In this paper we seek to monitor pedestrian-events (such as a person walking) on a footbridge using strain and acceleration data. The rate of this data acquisition and the number of sensing devices make the storage and analysis of this data a computational challenge. We introduce a streaming method to compress the sensor data, whils…

2019-04-12cs.LGstat.APstat.ML

Streaming PCA and Subspace Tracking: The Missing Data Case

University of Michigan（密歇根大学）；Carnegie Mellon University（卡内基梅隆大学）；Harvard University（哈佛大学）

For many modern applications in science and engineering, data are collected in a streaming fashion carrying time-varying information, and practitioners need to process them with a limited amount of memory and computational resources in a timely manner for decision making. This often is coupled with the missing data problem, such that only a small fraction of data attributes are observed. These complications impose significant, and unconventional, constraints on the problem of streaming Principal Component Analysis (PCA) and subspace tracking, which is an essential building block for many infer…

2018-06-12cs.ITcs.LGmath.IT

更多相关论文收起更多论文

CodeX: Bit-Flexible Encoding for Streaming-based FPGA Acceleration of DNNs

University of California San Diego（加利福尼亚大学圣地亚哥分校）

This paper proposes CodeX, an end-to-end framework that facilitates encoding, bitwidth customization, fine-tuning, and implementation of neural networks on FPGA platforms. CodeX incorporates nonlinear encoding to the computation flow of neural networks to save memory. The encoded features demand significantly lower storage compared to the raw full-precision activation values; therefore, the execution flow of CodeX hardware engine is completely performed within the FPGA using on-chip streaming buffers with no access to the off-chip DRAM. We further propose a fully-automated algorithm inspired b…

2019-01-17cs.LGstat.ML

Unsupervised Continual Learning in Streaming Environments

Nanyang Technological University（南洋理工大学）

A deep clustering network is desired for data streams because of its aptitude in extracting natural features thus bypassing the laborious feature engineering step. While automatic construction of the deep networks in streaming environments remains an open issue, it is also hindered by the expensive labeling cost of data streams rendering the increasing demand for unsupervised approaches. This paper presents an unsupervised approach of deep clustering network construction on the fly via simultaneous deep learning and clustering termed Autonomous Deep Clustering Network (ADCN). It combines the f…

2021-09-20cs.AIcs.LG

Auto-tuning Distributed Stream Processing Systems using Reinforcement Learning

Fine tuning distributed systems is considered to be a craftsmanship, relying on intuition and experience. This becomes even more challenging when the systems need to react in near real time, as streaming engines have to do to maintain pre-agreed service quality metrics. In this article, we present an automated approach that builds on a combination of supervised and reinforcement learning methods to recommend the most appropriate lever configurations based on previous load. With this, streaming engines can be automatically tuned without requiring a human to determine the right way and proper ti…

2018-09-14cs.DBcs.DCcs.LG

微通道中粘弹性流体的流电势介导电动输运的数据驱动设计优化

Data-Driven Design Optimization of Streaming-Potential-Mediated Electrokinetic Transport of Viscoelastic Fluids in Microchannels

Jadavpur University（jadavpur大学）；Heritage Institute of Technology（Heritage技术学院）

本研究开发代理辅助框架，结合机器学习代理模型与多目标优化策略，实现微通道中粘弹性流体电动输运设计的快速优化，可加速参数探索并为电动微流体装置提供设计指南。

2026-09-01cs.LGphysics.flu-dyn

通过概率稀疏化在低延迟特征引擎中解耦推理与状态更新

Decoupling Inference from State Updates in Low-Latency Feature Engines via Probabilistic Thinning

Feedzai

提出概率稀疏化方法，在流式机器学习管道中解耦推理与状态持久化，仅对信息事件触发状态更新，显著降低存储I/O和序列化开销，同时保持下游效用。

2026-06-16cs.DBcs.LG

数据存储中的动态适应：用于增强预取的实时机器学习

Dynamic Adaptation in Data Storage: Real-Time Machine Learning for Enhanced Prefetching

本研究提出一种集成流式分类模型的创新框架，用于预测多层存储系统中的文件访问模式（如下一文件偏移量），相比传统批量模型，显著提升了预测准确率、内存效率与系统适应性。

2025-01-28cs.DCcs.LGcs.OS

公共交通系统中显著延误变化的流式检测

Streaming detection of significant delay changes in public transport systems

Warsaw University of Technology（华沙理工大学）

针对公共交通延误影响出行选择的问题，提出基于流处理引擎的显著延误检测方法与参考架构，可在线识别反复出现的显著延误，经2000多辆车数据验证有效。

2024-04-11cs.LGphysics.soc-ph

Remaining Useful Life Prediction Using Temporal Deep Degradation Network for Complex Machinery with Attention-based Feature Extraction

Xi’an Jiaotong University（西安交通大学）；Xi’an Thermal Power Research Institute Co.,Ltd.（西安热工研究院有限公司）

The precise estimate of remaining useful life (RUL) is vital for the prognostic analysis and predictive maintenance that can significantly reduce failure rate and maintenance costs. The degradation-related features extracted from the sensor streaming data with neural networks can dramatically improve the accuracy of the RUL prediction. The Temporal deep degradation network (TDDN) model is proposed to make the RUL prediction with the degradation-related features given by the one-dimensional convolutional neural network (1D CNN) feature extraction and attention mechanism. 1D CNN is used to extra…

2022-02-21cs.AIcs.LGeess.SP

面向数字孪生应用中导数信息高斯过程代理的流式稀疏Cholesky方法

A Streaming Sparse Cholesky Method for Derivative-Informed Gaussian Process Surrogates Within Digital Twin Applications

organization= Department of Mechanical Engineering, University of Utah , addressline= , city= Salt Lake City , postcode= 84112 , state= UT , country= USA；organization= Kahlert School of Computing, University of Utah , city= Salt Lake City , postcode= 84112 , state= UT , country= USA；organization= Scientific Computing \& Imaging Institute, University of Utah , addressline= , city= Salt Lake City , postcode= 84112 , state= UT , country= USA

提出一种流式稀疏Cholesky方法，通过动态更新和导数信息增强高斯过程代理，降低协方差矩阵维度，实现数字孪生中飞机结构性能的实时预测。

2026-08-12cs.CEcs.LGstat.ML

Enhancing Dynamic Mode Decomposition Workflow with In-Situ Visualization and Data Compression

COPPE/Federal University of Rio de Janeiro（里约热内卢联邦大学COPPE研究生院）；Federal University of Juiz de Fora（茹伊斯迪福拉联邦大学）

Modern computational science and engineering applications are being improved by the advances in scientific machine learning. Data-driven methods such as Dynamic Mode Decomposition (DMD) can extract coherent structures from spatio-temporal data generated from dynamical systems and infer different scenarios for said systems. The spatio-temporal data comes as snapshots containing spatial information for each time instant. In modern engineering applications, the generation of high-dimensional snapshots can be time and/or resource-demanding. In the present study, we consider two strategies for enha…

2022-08-16cs.GRcs.LG