Streaming data are increasingly present in real-world applications such as sensor measurements, satellite data feed, stock market, and financial data. The main characteristics of these applications are the online arrival of data observations at high speed and the susceptibility to changes in the data distributions due to the dynamic nature of real environments. The data stream mining community still faces some primary challenges and difficulties related to the comparison and evaluation of new proposals, mainly due to the lack of publicly available non-stationary real-world datasets. The comparison of stream algorithms proposed in the literature is not an easy task, as authors do not always follow the same recommendations, experimental evaluation procedures, datasets, and assumptions. In this paper, we mitigate problems related to the choice of datasets in the experimental evaluation of stream classifiers and drift detectors. To that end, we propose a new public data repository for benchmarking stream algorithms with real-world data. This repository contains the most popular datasets from literature and new datasets related to a highly relevant public health problem that involves the recognition of disease vector insects using optical sensors. The main advantage of these new datasets is the prior knowledge of their characteristics and patterns of changes to evaluate new adaptive algorithm proposals adequately. We also present an in-depth discussion about the characteristics, reasons, and issues that lead to different types of changes in data distribution, as well as a critical review of common problems concerning the current benchmark datasets available in the literature.
A Framework for Evaluating and Benchmarking Concept Drift Detection Methods
University of Coimbra（科英布拉大学）；Victoria University of Wellington（惠灵顿维多利亚大学）；Commerzbank（德国商业银行）；University of Waikato（怀卡托大学）；AI Institute, University of Waikato（怀卡托大学人工智能研究所）
Towards Benchmarking Foundation Models for Tabular Data With Text
Department of Computer Science, University of Freiburg, Freiburg, Germany（弗赖堡大学计算机科学系）；Prior Labs, Freiburg, Germany（Prior实验室）；ELLIS Institute, Tubingen, Germany（图宾根ELLIS研究所）
MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion
Institute of Parallel and Distributed Systems (IPADS), Shanghai Jiao Tong University（上海交通大学并行与分布式系统研究所（IPADS））；Department of Precision Instrument, Tsinghua University（清华大学精密仪器系）；National Innovation Institute of High-end Smart Appliances（国家高端智能家电创新研究院）
Benchmarking World-Model Learning with Environment-Level Queries
Basis Research Institute；DFKI GmbH；Harvard University；Universit\'e de Montr\'eal \& Mila - Quebec AI Institute；University of Cambridge；Massachusetts Institute of Technology；Cornell University
RealBench: Benchmarking Verilog Generation Models with Real-World IP Designs
State Key Lab of Processors, Institute of Computing Technology, CAS（处理器国家重点实验室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；University of Science and Technology of China（中国科学技术大学）；Shanghai Innovation Center for Processor Technologies（上海处理器技术创新中心）；Cambricon Technologies（Cambricon技术公司）
Know Your Streams: On the Conceptualization, Characterization, and Generation of Intentional Event Streams
School of Engineering and Design, Technical University of Munich（慕尼黑工业大学工程与设计学院）；University of Bayreuth（拜罗伊特大学）；Christian-Albrechts-University Kiel（基尔大学）；Database Systems and Data Mining, Ludwig Maximilian University of Munich（慕尼黑大学数据库系统与数据挖掘）；Munich Center for Machine Learning（慕尼黑机器学习中心）
本文针对物联网和传感器驱动系统带来的连续实时事件流挑战，提出Stream of Intent原型生成器，用于生成可复现的意图事件流以提升流过程挖掘算法的适应性与准确性。
Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所信息处理实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Kuaishou Technology（快手科技）
