TS-MLLM: A Multi-Modal Large Language Model-based Framework for Industrial Time-Series Big Data Analysis
School of Automation Science and Electrical Engineering, Beihang University（北京航空航天大学自动化科学与电气工程学院）；School of Software, Beihang University（北京航空航天大学软件学院）；Hangzhou International Innovation Institute, Beihang University（北京航空航天大学杭州国际创新研究院）；State Key Laboratory of Intelligent Manufacturing System Technology（智能制造系统技术国家重点实验室）；School of Computer and Artificial Intelligence, Zhengzhou University（郑州大学计算机与人工智能学院）；Department of Computer Science, St. Francis Xavier University（圣弗朗西斯科大学计算机科学系）
Haiteng Wang, Yikang Li, Yunfei Zhu, Jingheng Yan, Lei Ren, Laurence T. Yang
Accurate analysis of industrial time-series big data is critical for the Prognostics and Health Management (PHM) of industrial equipment. While recent advancements in Large Language Models (LLMs) have shown promise in time-series analysis, existing methods typically focus on single-modality adaptations, failing to exploit the complementary nature of temporal signals, frequency-domain visual representations, and textual knowledge information. In this paper, we propose TS-MLLM, a unified multi-modal large language model framework designed to jointly model temporal signals, frequency-domain images, and textual domain knowledge. Specifically, we first develop an Industrial time-series Patch Modeling branch to capture long-range temporal dynamics. To integrate cross-modal priors, we introduce a Spectrum-aware Vision-Language Model Adaptation (SVLMA) mechanism that enables the model to internalize frequency-domain patterns and semantic context. Furthermore, a Temporal-centric Multi-modal Attention Fusion (TMAF) mechanism is designed to actively retrieve relevant visual and textual cues using temporal features as queries, ensuring deep cross-modal alignment. Extensive experiments on multiple industrial benchmarks demonstrate that TS-MLLM significantly outperforms state-of-the-art methods, particularly in few-shot and complex scenarios. The results validate our framework's superior robustness, efficiency, and generalization capabilities for industrial time-series prediction.
As the Industrial Internet of Things (IIoT) grows, systems are increasingly being monitored by arrays of sensors returning time-series data at ever-increasing 'volume, velocity and variety' (i.e. Industrial Big Data). An obvious use for these data is real-time systems condition monitoring and prognostic time to failure analysis (remaining useful life, RUL). (e.g. See white papers by Senseye.io, and output of the NASA Prognostics Center of Excellence (PCoE).) However, as noted by Agrawal and Choudhary 'Our ability to collect "big data" has greatly surpassed our capability to analyze it, undersc…
Blue Yonder GmbH（Blue Yonder公司）；University of Auckland（奥克兰大学）；University of Freiburg（弗莱堡大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
The all-relevant problem of feature selection is the identification of all strongly and weakly relevant attributes. This problem is especially hard to solve for time series classification and regression in industrial applications such as predictive maintenance or production line optimization, for which each label or regression target is associated with several time series and meta-information simultaneously. Here, we are proposing an efficient, scalable feature extraction algorithm for time series, which filters the available features in an early stage of the machine learning pipeline with res…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Given a multivariate big time series, can we detect anomalies as soon as they occur? Many existing works detect anomalies by learning how much a time series deviates away from what it should be in the reconstruction framework. However, most models have to cut the big time series into small pieces empirically since optimization algorithms cannot afford such a long series. The question is raised: do such cuts pollute the inherent semantic segments, like incorrect punctuation in sentences? Therefore, we propose a reconstruction-based anomaly detection method, MissGAN, iteratively learning to deco…
York University（约克大学）；iSecurity Consulting Inc（iSecurity咨询公司）；Dapasoft Inc（Dapasoft公司）
Industrial Information Technology (IT) infrastructures are often vulnerable to cyberattacks. To ensure security to the computer systems in an industrial environment, it is required to build effective intrusion detection systems to monitor the cyber-physical systems (e.g., computer networks) in the industry for malicious activities. This paper aims to build such intrusion detection systems to protect the computer networks from cyberattacks. More specifically, we propose a novel unsupervised machine learning approach that combines the K-Means algorithm with the Isolation Forest for anomaly detec…
A rich supply of data and innovative algorithms have made data-driven modeling a popular technique in modern industry. Among various data-driven methods, latent variable models (LVMs) and their counterparts account for a major share and play a vital role in many industrial modeling areas. LVM can be generally divided into statistical learning-based classic LVM and neural networks-based deep LVM (DLVM). We first discuss the definitions, theories and applications of classic LVMs in detail, which serves as both a comprehensive tutorial and a brief application survey on classic LVMs. Then we prese…
How to Do Machine Learning with Small Data? -- A Review from an Industrial Perspective
Fraunhofer Institute for Ceramic Technologies and Systems（弗劳恩霍夫陶瓷技术与系统研究所）；Brandenburg University of Technology Cottbus–Senftenberg（勃兰登堡工业大学科特布斯-森夫滕贝格）
Due to its probabilistic nature, fault prognostics is a prime example of a use case for deep learning utilizing big data. However, the low availability of such data sets combined with the high effort of fitting, parameterizing and evaluating complex learning algorithms to the heterogenous and dynamic settings typical for industrial applications oftentimes prevents the practical application of this approach. Automatic adaptation to new or dynamically changing fault prognostics scenarios can be achieved using transfer learning or continual learning methods. In this paper, a first survey of such …
University of Luxembourg（卢森堡大学）；University of Quebec in Montreal（蒙特利尔魁北克大学）；CHOICE Technologies Holding Sàrl（CHOICE科技控股有限公司）
In machine learning, a bias occurs whenever training sets are not representative for the test data, which results in unreliable models. The most common biases in data are arguably class imbalance and covariate shift. In this work, we aim to shed light on this topic in order to increase the overall attention to this issue in the field of machine learning. We propose a scalable novel framework for reducing multiple biases in high-dimensional data sets in order to train more reliable predictors. We apply our methodology to the detection of irregular power usage from real, noisy industrial data. I…
Table (database) / Relational database Classification for big/smart/fast data machine learning is one of the most important tasks of predictive analytics and extracting valuable information from data. It is core applied technique for what now understood under data science and/or artificial intelligence. Widely used Decision Tree (Random Forest) and rare used rule based PRISM , VFST, etc classifiers are empirical substitutions of theoretically correct to use Boolean functions minimization. Developing Minimization of Boolean functions algorithms is started long time ago by Edward Veitch's 1952. …
Federated Learning in Big Model Era: Domain-Specific Multimodal Large Models
ENN Group Co.,Ltd.（ENN集团有限公司）；China University of Petroleum（中国石油大学）；Unicom (Shanghai) Industrial Internet Co.,Ltd.（联通（上海）工业互联网有限公司）；Qingdao Windaka Technology Co.,Ltd.（青岛Windaka科技有限公司）
Carnegie Mellon University（卡内基梅隆大学）；Institute for Infocomm Research, A*STAR（新加坡科技研究局信息通信研究所）
In real world industrial applications of topic modeling, the ability to capture gigantic conceptual space by learning an ultra-high dimensional topical representation, i.e., the so-called "big model", is becoming the next desideratum after enthusiasms on "big data", especially for fine-grained downstream tasks such as online advertising, where good performances are usually achieved by regression-based predictors built on millions if not billions of input features. The conventional data-parallel approach for training gigantic topic models turns out to be rather inefficient in utilizing the powe…
Clemson University（克莱姆森大学）；AI Research Institute for Science and Engineering (AIRISE), Clemson University（克莱姆森大学人工智能科学与工程研究院）
Artificial intelligence (AI) is paving the way towards the fourth industrial revolution with the fire domain (Fire 4.0). As a matter of fact, the next few years will be elemental to how this technology will shape our academia, practice, and entrepreneurship. Despite the growing interest between fire research groups, AI remains absent of our curriculum, and we continue to lack a methodical framework to adopt, apply and create AI solutions suitable for our problems. The above is also true for parallel engineering domains (i.e., civil/mechanical engineering), and in order to negate the notion of …
The rise of Big Data has led to new demands for Machine Learning (ML) systems to learn complex models with millions to billions of parameters, that promise adequate capacity to digest massive datasets and offer powerful predictive analytics thereupon. In order to run ML algorithms at such scales, on a distributed cluster with 10s to 1000s of machines, it is often the case that significant engineering efforts are required --- and one might fairly ask if such engineering truly falls within the domain of ML research or not. Taking the view that Big ML systems can benefit greatly from ML-rooted st…
Fashion is now among the largest industries worldwide, for it represents human history and helps tell the worlds story. As a result of the Fourth Industrial Revolution, the Internet has become an increasingly important source of fashion information. However, with a growing number of web pages and social data, it is nearly impossible for humans to manually catch up with the ongoing evolution and the continuously variable content in this domain. The proper management and exploitation of big data can pave the way for the substantial growth of the global economy as well as citizen satisfaction. Th…
Siemens AG（西门子股份公司）；Technical University of Munich（慕尼黑工业大学）
Focusing on comprehensive networking, big data, and artificial intelligence, the Industrial Internet-of-Things (IIoT) facilitates efficiency and robustness in factory operations. Various sensors and field devices play a central role, as they generate a vast amount of real-time data that can provide insights into manufacturing. The synergy of complex event processing (CEP) and machine learning (ML) has been developed actively in the last years in IIoT to identify patterns in heterogeneous data streams and fuse raw data into tangible facts. In a traditional compute-centric paradigm, the raw fiel…
2026 Roadmap on Artificial Intelligence and Machine Learning for Smart Manufacturing
Center for Industrial Artificial Intelligence, Department of Mechanical Engineering, University of Maryland, College Park（工业人工智能中心，机械工程系，马里兰大学College Park分校）；Department of Management, Economics and Industrial Engineering, Politecnico di Milano（管理、经济与工业工程系，米兰理工学院）；Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University（工业与系统工程系，香港理工大学）；Centre for Advanced Manufacturing & Supply Chains, World Economic Forum（先进制造与供应链研究中心，世界经济论坛）；Department of Mechanical Engineering, Indian Institute of Technology Indore（机械工程系，印度理工学院Indore分校）；Future Information Innovative College, Fudan University（未来信息创新学院，复旦大学）；Department of Mechanical Engineering, Seoul National University（机械工程系，首尔国立大学）；Department of Mechanical and Information Engineering, University of Seoul（机械与信息工程系，首尔大学）；Onepredict Corp.（Onepredict公司）；School of Mechanical and Aerospace Engineering, Nanyang Technological University（机械与航空航天工程学院，南洋理工大学）；Singapore Centre for 3D Printing, Nanyang Technological University（新加坡3D打印中心，南洋理工大学）；Mechanical Engineering, Purdue University（机械工程系，普渡大学）；Digital Twin International Research Center, International Institute for Interdisciplinary and Frontiers, Beihang University（数字孪生国际研究中心， interdisciplinary and Frontiers 国际研究院，北京航空航天大学）；School of Automation Science and Electrical Engineering, Beihang University（自动化科学与电气工程学院，北京航空航天大学）；Department of Engineering, University of Cambridge（工程系，剑桥大学）；Center for Advanced Manufacturing, University of Southern California（先进制造中心，南加州大学）；School of Sustainability Engineering and Environmental Engineering, Purdue University（可持续工程与环境工程系，普渡大学）；School of Mechanical Engineering, Sungkyunkwan University（机械工程系，全南大学）；Intelligent Maintenance and Operations Systems, EPFL（智能维护与运营系统，苏黎世联邦理工学院）；Department of Mechanical Engineering, Massachusetts Institute of Technology（机械工程系，麻省理工学院）；J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University（J. Mike Walker ’66 机械工程系，德克萨斯A&M大学）；Department of Mechanical and Process Engineering, ETH Zürich（机械与工艺工程系，苏黎世联邦理工学院）
IBM T.J. Watson Research Center（IBM T.J. 沃森研究中心）；IBM Research（IBM研究院）
An automated machine learning framework for geospatial data named PAIRS AutoGeo is introduced on IBM PAIRS Geoscope big data and analytics platform. The framework simplifies the development of industrial machine learning solutions leveraging geospatial data to the extent that the user inputs are minimized to merely a text file containing labeled GPS coordinates. PAIRS AutoGeo automatically gathers required data at the location coordinates, assembles the training data, performs quality check, and trains multiple machine learning models for subsequent deployment. The framework is validated using…
Microsoft（微软）；IT University of Copenhagen（哥本哈根信息技术大学）；Pinecone；Meta AI Research（Meta AI研究院）；Carnegie Mellon University（卡内基梅隆大学）；Yandex；Zilliz；University of Maryland（马里兰大学）；Fudan University（复旦大学）；Shanghai Jiao Tong University（上海交通大学）；Baidu（百度）
本文总结了 NeurIPS 2023 Big ANN Challenge 竞赛，该竞赛聚焦过滤搜索、分布外数据及稀疏与流式变体等复杂 ANN 搜索，参赛者在资源受限下取得显著性能提升，推动了该领域发展。
Over the last two decades, we have seen extensive industrial research in the area of computational advertising. In this paper, our goal is to study the performance of various online learning algorithms to identify and display the best ads/offers with the highest conversion rates to web users. We formulate our ad-selection problem as a Multi-Armed Bandit problem which is a classical paradigm in Machine Learning. We have been applying machine learning, data mining, probability, and statistics to analyze big data in the ad-tech space and devise efficient ad selection strategies. This article high…
University of Trento（特伦托大学）；University of Twente（特温特大学）
Many industrial sectors have been collecting big sensor data. With recent technologies for processing big data, companies can exploit this for automatic failure detection and prevention. We propose the first completely automated method for failure analysis, machine-learning fault trees from raw observational data with continuous variables. Our method scales well and is tested on a real-world, five-year dataset of domestic heater operations in The Netherlands, with 31 million unique heater-day readings, each containing 27 sensor and 11 failure variables. Our method builds on two previous proced…
Microsoft Research（微软研究院）；Beihang University（北京航空航天大学）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）；Carnegie Mellon University（卡内基梅隆大学）
When building large-scale machine learning (ML) programs, such as big topic models or deep neural nets, one usually assumes such tasks can only be attempted with industrial-sized clusters with thousands of nodes, which are out of reach for most practitioners or academic researchers. We consider this challenge in the context of topic modeling on web-scale corpora, and show that with a modest cluster of as few as 8 machines, we can train a topic model with 1 million topics and a 1-million-word vocabulary (for a total of 1 trillion parameters), on a document collection with 200 billion tokens -- …
