This paper introduced a reinforcement learning based decision support system in textile manufacturing process. A solution optimization problem of color fading ozonation is discussed and set up as a Markov Decision Process (MDP) in terms of tuple {S, A, P, R}. Q-learning is used to train an agent in the interaction with the setup environment by accumulating the reward R. According to the application result, it is found that the proposed MDP model has well expressed the optimization problem of textile manufacturing process discussed in this paper, therefore the use of reinforcement learning to support decision making in this sector is conducted and proven that is applicable with promising prospects.
Textile manufacturing is a typical traditional industry involving high complexity in interconnected processes with limited capacity on the application of modern technologies. Decision-making in this domain generally takes multiple criteria into consideration, which usually arouses more complexity. To address this issue, the present paper proposes a decision support system that combines the intelligent data-based random forest (RF) models and a human knowledge based analytical hierarchical process (AHP) multi-criteria structure in accordance to the objective and the subjective factors of the te…
Toolpath optimization of metal-based additive manufacturing processes is currently hampered by the high-dimensionality of its design space. In this work, a reinforcement learning platform is proposed that dynamically learns toolpath strategies to build an arbitrary part. To this end, three prominent model-free reinforcement learning formulations are investigated to design additive manufacturing toolpaths and demonstrated for two cases of dense and sparse reward structures. The results indicate that this learning-based toolpath design approach achieves high scores, especially when a dense rewar…
Efficient dispatching rule in manufacturing industry is key to ensure product on-time delivery and minimum past-due and inventory cost. Manufacturing, especially in the developed world, is moving towards on-demand manufacturing meaning a high mix, low volume product mix. This requires efficient dispatching that can work in dynamic and stochastic environments, meaning it allows for quick response to new orders received and can work over a disparate set of shop floor settings. In this paper we address this problem of dispatching in manufacturing. Using reinforcement learning (RL), we propose a n…
University of Klagenfurt（克拉根福大学）；Graz University of Technology（格拉茨技术大学）；Infineon Technologies（英飞凌科技）
Semiconductor manufacturing is a notoriously complex and costly multi-step process involving a long sequence of operations on expensive and quantity-limited equipment. Recent chip shortages and their impacts have highlighted the importance of semiconductors in the global supply chains and how reliant on those our daily lives are. Due to the investment cost, environmental impact, and time scale needed to build new factories, it is difficult to ramp up production when demand spikes. This work introduces a method to successfully learn to schedule a semiconductor manufacturing facility more effici…
Design for Manufacturing: A Manufacturability Knowledge-Integrated Reinforcement Learning Framework for Free-Form Pipe Routing in Aeroengines
State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University（浙江大学流体动力与机电系统国家重点实验室）；Engineering Research Center for Design Engineering and Digital Twin of Zhejiang Province, Zhejiang University（浙江省设计工程与数字孪生工程研究中心）；Zhejiang Changxing Heliang Intelligent Equipment Co., Ltd.（浙江长兴鹤浪智能装备有限公司）
South Westphalia University of Applied Sciences（西南法伦应用科学大学）
This paper introduces a novel combination of scheduling control on a flexible robot manufacturing cell with curiosity based reinforcement learning. Reinforcement learning has proved to be highly successful in solving tasks like robotics and scheduling. But this requires hand tuning of rewards in problem domains like robotics and scheduling even where the solution is not obvious. To this end, we apply a curiosity based reinforcement learning, using intrinsic motivation as a form of reward, on a flexible robot manufacturing cell to alleviate this problem. Further, the learning agents are embedde…
University of Piraeus（比雷埃夫斯大学）；Jožef Stefan Institute（约泽夫·斯特凡研究所）；Jožef Stefan International Postgraduate School（约泽夫·斯特凡国际研究生院）
The implementation of Artificial Intelligence (AI) systems in the manufacturing domain enables higher production efficiency, outstanding performance, and safer operations, leveraging powerful tools such as deep learning and reinforcement learning techniques. Despite the high accuracy of these models, they are mostly considered black boxes: they are unintelligible to the human. Opaqueness affects trust in the system, a factor that is critical in the context of decision-making. We present an overview of Explainable Artificial Intelligence (XAI) techniques as a means of boosting the transparency …
Driven by the key challenges of cell therapy manufacturing, including high complexity, high uncertainty, and very limited process observations, we propose a hybrid model-based reinforcement learning (RL) to efficiently guide process control. We first create a probabilistic knowledge graph (KG) hybrid model characterizing the risk- and science-based understanding of biomanufacturing process mechanisms and quantifying inherent stochasticity, e.g., batch-to-batch variation. It can capture the key features, including nonlinear reactions, nonstationary dynamics, and partially observed state. This h…
University of Toronto（多伦多大学）；University of Alberta（阿尔伯塔大学）；Shanghai Jiao Tong University（上海交通大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）
Traditional biological and pharmaceutical manufacturing plants are controlled by human workers or pre-defined thresholds. Modernized factories have advanced process control algorithms such as model predictive control (MPC). However, there is little exploration of applying deep reinforcement learning to control manufacturing plants. One of the reasons is the lack of high fidelity simulations and standard APIs for benchmarking. To bridge this gap, we develop an easy-to-use library that includes five high-fidelity simulation environments: BeerFMTEnv, ReactorEnv, AtropineEnv, PenSimEnv and mAbEnv,…
Reinforcement Learning for Synchronised Flow Control in a Dual-Gate Resin Infusion System
Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, HUMAN-tech, Universitat Politècnica de València（人类中心技术大学研究院，HUMAN-tech，瓦伦西亚理工大学）；Department of Civil and Mechanical Engineering, Technical University of Denmark（土木与机械工程系，技术大学 of Denmark）；Artikode Intelligence S.L
Energy-Efficient Prediction in Textile Manufacturing: Enhancing Accuracy and Data Efficiency With Ensemble Deep Transfer Learning
Department of Electrical Engineering, National Tsing Hua University（国立清华大学电子工程系）；School of Information Technology, Faculty of Science, Engineering and Built Environment, Deakin University（德克萨斯大学信息科技学院）；Industrial Technology Research Institute（工业技术研究所）
Virginia Tech（弗吉尼亚理工学院暨州立大学）；New Jersey Institute of Technology（新泽西理工学院）；IoTeX
Additive Manufacturing (AM) is a powerful technology that produces complex 3D geometries using various materials in a layer-by-layer fashion. However, quality assurance is the main challenge in AM industry due to the possible time-varying processing conditions during AM process. Notably, new defects may occur during printing, which cannot be mitigated by offline analysis tools that focus on existing defects. This challenge motivates this work to develop online learning-based methods to deal with the new defects during printing. Since AM typically fabricates a small number of customized product…
Duke University（杜克大学）；Fidelity Investments（富达投资）；Cornell University（康奈尔大学）
Flexibility design problems are a class of problems that appear in strategic decision-making across industries, where the objective is to design a ($e.g.$, manufacturing) network that affords flexibility and adaptivity. The underlying combinatorial nature and stochastic objectives make flexibility design problems challenging for standard optimization methods. In this paper, we develop a reinforcement learning (RL) framework for flexibility design problems. Specifically, we carefully design mechanisms with noisy exploration and variance reduction to ensure empirical success and show the unique …
TH Köln – University of Applied Sciences（科隆应用技术大学）；Cologne Cobots Lab（科隆协作机器人实验室）；Laboratory of Manufacturing Systems（制造系统实验室）；Institute of Telecommunications Technology（电信技术研究所）
Individualized manufacturing is becoming an important approach as a means to fulfill increasingly diverse and specific consumer requirements and expectations. While there are various solutions to the implementation of the manufacturing process, such as additive manufacturing, the subsequent automated assembly remains a challenging task. As an approach to this problem, we aim to teach a collaborative robot to successfully perform pick and place tasks by implementing reinforcement learning. For the assembly of an individualized product in a constantly changing manufacturing environment, the simu…
Process optimization for metal additive manufacturing (AM) is crucial to ensure repeatability, control microstructure, and minimize defects. Despite efforts to address this via the traditional design of experiments and statistical process mapping, there is limited insight on an on-the-fly optimization framework that can be integrated into a metal AM system. Additionally, most of these methods, being data-intensive, cannot be supported by a metal AM alloy or system due to budget restrictions. To tackle this issue, the article introduces a Reinforcement Learning (RL) methodology transformed into…
Model-Free Adaptive Optimal Control of Episodic Fixed-Horizon Manufacturing Processes using Reinforcement Learning
Institute Intelligent Systems Research Group, Karlsruhe University of Applied Sciences（智能系统研究组，卡尔斯鲁厄应用科学大学）；Institute for Applied Materials (IAM-CMS), Karlsruhe Institute of Technology（应用材料研究所（IAM-CMS），卡尔斯鲁厄理工学院）
Logic-informed reinforcement learning for cross-domain optimization of large-scale cyber-physical systems
State Key Laboratory of Robotics and Intelligent Systems（机器人与智能系统国家重点实验室）；Shenyang Institute of Automation（沈阳自动化研究所）；Chinese Academy of Sciences（中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
MFRL-BI: Design of a Model-free Reinforcement Learning Process Control Scheme by Using Bayesian Inference
Antai College of Economics and Management, Shanghai Jiao Tong University（上海交通大学安泰经济与管理学院）；Smart Manufacturing Thrust, Systems Hub, The Hong Kong University of Science and Technology（香港科技大学系统枢纽智能制造方向）
Laser Scan Path Design for Controlled Microstructure in Additive Manufacturing with Integrated Reduced-Order Phase-Field Modeling and Deep Reinforcement Learning
Computational Science Program, The University of Texas at El Paso（德克萨斯理工大学计算科学计划）；Department of Aerospace and Mechanical Engineering, The University of Texas at El Paso（德克萨斯理工大学航空航天与机械工程系）；School of Manufacturing Systems and Networks, Arizona State University（亚利桑那州立大学制造系统与网络学院）
Review of Cloud Service Composition for Intelligent Manufacturing
School of Cyber Science and Engineering, Zhengzhou University（郑州大学网络空间安全学院）；School of Electrical and Information Engineering, Zhengzhou University（郑州大学电气与信息工程学院）；Department of Automation, Tsinghua University（清华大学自动化系）
Digital Twin Synchronization: Bridging the Sim-RL Agent to a Real-Time Robotic Additive Manufacturing Control
Department of Mechanical Engineering, University of Louisiana at Lafayette（路易斯安那大学拉斐特分校机械工程系）；Computer Science & Information Systems Department, Bradley University（布雷纳德大学计算机科学与信息系统系）
Knitting Robots: A Deep Learning Approach for Reverse-Engineering Fabric Patterns
Bharti School of Engineering and Computer Science, Laurentian University（劳伦森大学巴蒂工程与计算机科学学院）；Shanghai International Fashion Education Centre（上海国际时尚教育中心）
Constrained Reinforcement Learning for Dynamic Material Handling
Research Institute of Trustworthy Autonomous Systems (RITAS), Southern University of Science and Technology（南方科技大学可信自主系统研究院）；Southern University of Science and Technology（南方科技大学）；Trustworthiness Theory Research Center, Huawei Technologies Co., Ltd（华为技术有限公司可信理论研究中心）
In this paper we apply guided policy search (GPS) based reinforcement learning framework for a high dimensional optimal control problem arising in an additive manufacturing process. The problem comprises of controlling the process parameters so that layer-wise deposition of material leads to desired geometric characteristics of the resulting part surface while minimizing the material deposited. A realistic simulation model of the deposition process along with carefully selected set of guiding distributions generated based on iterative Linear Quadratic Regulator is used to train a neural networ…
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；School of Intelligent Systems Science and Engineering, Jinan University（暨南大学智能系统科学与工程学院）
The dynamic job-shop scheduling problem (DJSP) is a class of scheduling tasks that specifically consider the inherent uncertainties such as changing order requirements and possible machine breakdown in realistic smart manufacturing settings. Since traditional methods cannot dynamically generate effective scheduling strategies in face of the disturbance of environments, we formulate the DJSP as a Markov decision process (MDP) to be tackled by reinforcement learning (RL). For this purpose, we propose a flexible hybrid framework that takes disjunctive graphs as states and a set of general dispatc…
Dongeui University（东义大学）；Pusan National University（釜山国立大学）
Process mining is a methodology for the derivation and analysis of process models based on the event log. When process mining is employed to analyze business processes, the process discovery step, the conformance checking step, and the enhancements step are repeated. If a user wants to analyze a process from multiple perspectives (such as activity perspectives, originator perspectives, and time perspectives), the above procedure, inconveniently, has to be repeated over and over again. Although past studies involving process mining have applied detailed stepwise methodologies, no attempt has be…
Multi-Head Attention-Based Feature Extractor Integration with Soft Actor-Critic for Porosity Prediction and Process Parameter Optimization in Additive Manufacturing
Department of Electrical Engineering, Amirkabir University of Technology (Tehran Polytechnic)（阿米尔卡比尔理工大学（德黑兰理工大学）电气工程系）；Department of Mechanical Engineering, Amirkabir University of Technology (Tehran Polytechnic)（阿米尔卡比尔理工大学（德黑兰理工大学）机械工程系）；School of Computer Science, University of Birmingham（伯明翰大学计算机科学学院）
Hitachi America Ltd.（日立美国有限公司）；Stevens Institute of Technology（史蒂文斯理工学院）
Dynamic dispatching is one of the core problems for operation optimization in traditional industries such as mining, as it is about how to smartly allocate the right resources to the right place at the right time. Conventionally, the industry relies on heuristics or even human intuitions which are often short-sighted and sub-optimal solutions. Leveraging the power of AI and Internet of Things (IoT), data-driven automation is reshaping this area. However, facing its own challenges such as large-scale and heterogenous trucks running in a highly dynamic environment, it can barely adopt methods de…
Northeastern University（东北大学）；University of Waterloo（滑铁卢大学）
Biopharmaceutical manufacturing faces critical challenges, including complexity, high variability, lengthy lead time, and limited historical data and knowledge of the underlying system stochastic process. To address these challenges, we propose a green simulation assisted model-based reinforcement learning to support process online learning and guide dynamic decision making. Basically, the process model risk is quantified by the posterior distribution. At any given policy, we predict the expected system response with prediction risk accounting for both inherent stochastic uncertainty and model…
Learning to Optimize Capacity Planning in Semiconductor Manufacturing
College of Computing and Data Science, Nanyang Technological University, Singapore（计算与数据科学学院，南洋理工大学，新加坡）；D-SIMLAB Technologies Pte Ltd, Singapore（D-SIMLAB技术私人有限公司，新加坡）
Smart Manufacturing: MLOps-Enabled Event-Driven Architecture for Enhanced Control in Steel Production
Karlstad University, Computer Science Department（卡尔斯塔德大学计算机科学系）；University of Bologna, Department of Computer Science and Engineering（博洛尼亚大学计算机科学与工程系）；Deggendorf Institute of Technology, Faculty of Applied Computer Science（德格多夫技术学院应用计算机科学系）；Smart Industrial Automation, RISE Research Institutes of Sweden（智能工业自动化，瑞典RISE研究机构）
Reinforcement learning (RL) is one of the three basic paradigms of machine learning. It has demonstrated impressive performance in many complex tasks like Go and StarCraft, which is increasingly involved in smart manufacturing and autonomous driving. However, RL consistently suffers from the exploration-exploitation dilemma. In this paper, we investigated the problem of improving exploration in RL and introduced the intrinsically-motivated RL. In sharp contrast to the classic exploration strategies, intrinsically-motivated RL utilizes the intrinsic learning motivation to provide sustainable ex…
In manufacturing, assembly tasks have been a challenge for learning algorithms due to variant dynamics of different environments. Reinforcement learning (RL) is a promising framework to automatically learn these tasks, yet it is still not easy to apply a learned policy or skill, that is the ability of solving a task, to a similar environment even if the deployment conditions are only slightly different. In this paper, we address the challenge of transferring knowledge within a family of similar tasks by leveraging multiple skill priors. We propose to learn prior distribution over the specific …
Multi-objective optimizations are frequently encountered in engineering practices. The solution techniques and parametric selections however are usually problem-specific. In this study we formulate a reinforcement learning hyper-heuristic scheme, and propose four low-level heuristics which can work coherently with the single point search algorithm MOSA/R (Multi-Objective Simulated Annealing Algorithm based on Re-seed) towards multi-objective optimization problems of general applications. Making use of the domination amount, crowding distance and hypervolume calculations, the proposed hyper-heu…
Industrial robots are widely used in various manufacturing environments due to their efficiency in doing repetitive tasks such as assembly or welding. A common problem for these applications is to reach a destination without colliding with obstacles or other robot arms. Commonly used sampling-based path planning approaches such as RRT require long computation times, especially in complex environments. Furthermore, the environment in which they are employed needs to be known beforehand. When utilizing the approaches in new environments, a tedious engineering effort in setting hyperparameters ne…
Multi-Mode Process Control Using Multi-Task Inverse Reinforcement Learning
State Key Laboratory of Industrial Control Technology, Institute of Cyber-Systems and Control, Zhejiang University（工业控制技术国家重点实验室，系统与控制研究院，浙江大学）；Department of Chemical Engineering, Chung-Yuan Christian University（化学工程系， Chung-Yuan Christian University）；Department of Chemical and Materials Engineering, University of Alberta（化学与材料工程系，阿尔伯塔大学）
Optimizing Job Shop Scheduling in the Furniture Industry: A Reinforcement Learning Approach Considering Machine Setup, Batch Variability, and Intralogistics
Faculty of Wood Technology and Construction（木材技术与建筑学院）；Rosenheim Technical University of Applied Sciences（罗森heim应用技术大学）；Center for Research, Development and Transfer（研究、开发与转让中心）；Faculty of Management and Engineering（管理与工程学院）
