Multi-objective optimization of the textile manufacturing process is an increasing challenge because of the growing complexity involved in the development of the textile industry. The use of intelligent techniques has been often discussed in this domain, although a significant improvement from certain successful applications has been reported, the traditional methods failed to work with high-as well as human intervention. Upon which, this paper proposed a multi-agent reinforcement learning (MARL) framework to transform the optimization process into a stochastic game and introduced the deep Q-networks algorithm to train the multiple agents. A utilitarian selection mechanism was employed in the stochastic game, which (-greedy policy) in each state to avoid the interruption of multiple equilibria and achieve the correlated equilibrium optimal solutions of the optimizing process. The case study result reflects that the proposed MARL system is possible to achieve the optimal solutions for the textile ozonation process and it performs better than the traditional approaches.
Textile manufacturing is a typical traditional industry involving high complexity in interconnected processes with limited capacity on the application of modern technologies. Decision-making in this domain generally takes multiple criteria into consideration, which usually arouses more complexity. To address this issue, the present paper proposes a decision support system that combines the intelligent data-based random forest (RF) models and a human knowledge based analytical hierarchical process (AHP) multi-criteria structure in accordance to the objective and the subjective factors of the te…
This paper introduced a reinforcement learning based decision support system in textile manufacturing process. A solution optimization problem of color fading ozonation is discussed and set up as a Markov Decision Process (MDP) in terms of tuple {S, A, P, R}. Q-learning is used to train an agent in the interaction with the setup environment by accumulating the reward R. According to the application result, it is found that the proposed MDP model has well expressed the optimization problem of textile manufacturing process discussed in this paper, therefore the use of reinforcement learning to s…
Coordinating from Memory: Graph-Structured Experience Reuse for Multi-Agent Adaptation in Dynamic Manufacturing
School of Computer Science, University of Sydney（悉尼大学计算机科学学院）；Khoury College of Computer Sciences, Northeastern University（美国东北大学库里计算机科学学院）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；School of Life and Environmental Sciences, University of Sydney（悉尼大学生命与环境科学学院）；College of Business and Economics, Australian National University（澳大利亚国立大学商业与经济学院）
South Westphalia University of Applied Sciences（西南法伦应用科学大学）
This paper introduces a novel combination of scheduling control on a flexible robot manufacturing cell with curiosity based reinforcement learning. Reinforcement learning has proved to be highly successful in solving tasks like robotics and scheduling. But this requires hand tuning of rewards in problem domains like robotics and scheduling even where the solution is not obvious. To this end, we apply a curiosity based reinforcement learning, using intrinsic motivation as a form of reward, on a flexible robot manufacturing cell to alleviate this problem. Further, the learning agents are embedde…
Toolpath optimization of metal-based additive manufacturing processes is currently hampered by the high-dimensionality of its design space. In this work, a reinforcement learning platform is proposed that dynamically learns toolpath strategies to build an arbitrary part. To this end, three prominent model-free reinforcement learning formulations are investigated to design additive manufacturing toolpaths and demonstrated for two cases of dense and sparse reward structures. The results indicate that this learning-based toolpath design approach achieves high scores, especially when a dense rewar…
University of Bologna（博洛尼亚大学）；University of Padova（帕多瓦大学）；Aalborg University（奥尔堡大学）
The automation of factories and manufacturing processes has been accelerating over the past few years, boosted by the Industry 4.0 paradigm, including diverse scenarios with mobile, flexible agents. Efficient coordination between mobile robots requires reliable wireless transmission in highly dynamic environments, often with strict timing requirements. Goal-oriented communication is a possible solution for this problem: communication decisions should be optimized for the target control task, providing the information that is most relevant to decide which action to take. From the control perspe…
Efficient dispatching rule in manufacturing industry is key to ensure product on-time delivery and minimum past-due and inventory cost. Manufacturing, especially in the developed world, is moving towards on-demand manufacturing meaning a high mix, low volume product mix. This requires efficient dispatching that can work in dynamic and stochastic environments, meaning it allows for quick response to new orders received and can work over a disparate set of shop floor settings. In this paper we address this problem of dispatching in manufacturing. Using reinforcement learning (RL), we propose a n…
University of Klagenfurt（克拉根福大学）；Graz University of Technology（格拉茨技术大学）；Infineon Technologies（英飞凌科技）
Semiconductor manufacturing is a notoriously complex and costly multi-step process involving a long sequence of operations on expensive and quantity-limited equipment. Recent chip shortages and their impacts have highlighted the importance of semiconductors in the global supply chains and how reliant on those our daily lives are. Due to the investment cost, environmental impact, and time scale needed to build new factories, it is difficult to ramp up production when demand spikes. This work introduces a method to successfully learn to schedule a semiconductor manufacturing facility more effici…
Design for Manufacturing: A Manufacturability Knowledge-Integrated Reinforcement Learning Framework for Free-Form Pipe Routing in Aeroengines
State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University（浙江大学流体动力与机电系统国家重点实验室）；Engineering Research Center for Design Engineering and Digital Twin of Zhejiang Province, Zhejiang University（浙江省设计工程与数字孪生工程研究中心）；Zhejiang Changxing Heliang Intelligent Equipment Co., Ltd.（浙江长兴鹤浪智能装备有限公司）
Model-Free Adaptive Optimal Control of Episodic Fixed-Horizon Manufacturing Processes using Reinforcement Learning
Institute Intelligent Systems Research Group, Karlsruhe University of Applied Sciences（智能系统研究组，卡尔斯鲁厄应用科学大学）；Institute for Applied Materials (IAM-CMS), Karlsruhe Institute of Technology（应用材料研究所（IAM-CMS），卡尔斯鲁厄理工学院）
Process optimization for metal additive manufacturing (AM) is crucial to ensure repeatability, control microstructure, and minimize defects. Despite efforts to address this via the traditional design of experiments and statistical process mapping, there is limited insight on an on-the-fly optimization framework that can be integrated into a metal AM system. Additionally, most of these methods, being data-intensive, cannot be supported by a metal AM alloy or system due to budget restrictions. To tackle this issue, the article introduces a Reinforcement Learning (RL) methodology transformed into…
Review of Cloud Service Composition for Intelligent Manufacturing
School of Cyber Science and Engineering, Zhengzhou University（郑州大学网络空间安全学院）；School of Electrical and Information Engineering, Zhengzhou University（郑州大学电气与信息工程学院）；Department of Automation, Tsinghua University（清华大学自动化系）
Digital Twin Synchronization: Bridging the Sim-RL Agent to a Real-Time Robotic Additive Manufacturing Control
Department of Mechanical Engineering, University of Louisiana at Lafayette（路易斯安那大学拉斐特分校机械工程系）；Computer Science & Information Systems Department, Bradley University（布雷纳德大学计算机科学与信息系统系）
Multi-Head Attention-Based Feature Extractor Integration with Soft Actor-Critic for Porosity Prediction and Process Parameter Optimization in Additive Manufacturing
Department of Electrical Engineering, Amirkabir University of Technology (Tehran Polytechnic)（阿米尔卡比尔理工大学（德黑兰理工大学）电气工程系）；Department of Mechanical Engineering, Amirkabir University of Technology (Tehran Polytechnic)（阿米尔卡比尔理工大学（德黑兰理工大学）机械工程系）；School of Computer Science, University of Birmingham（伯明翰大学计算机科学学院）
Reinforcement Learning for Synchronised Flow Control in a Dual-Gate Resin Infusion System
Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, HUMAN-tech, Universitat Politècnica de València（人类中心技术大学研究院，HUMAN-tech，瓦伦西亚理工大学）；Department of Civil and Mechanical Engineering, Technical University of Denmark（土木与机械工程系，技术大学 of Denmark）；Artikode Intelligence S.L
University of Oxford（牛津大学）；Imperial College London（帝国理工学院）
Scheduling plays an important role in automated production. Its impact can be found in various fields such as the manufacturing industry, the service industry and the technology industry. A scheduling problem (NP-hard) is a task of finding a sequence of job assignments on a given set of machines with the goal of optimizing the objective defined. Methods such as Operation Research, Dispatching Rules, and Combinatorial Optimization have been applied to scheduling problems but no solution guarantees to find the optimal solution. The recent development of Reinforcement Learning has shown success i…
Duke University（杜克大学）；Fidelity Investments（富达投资）；Cornell University（康奈尔大学）
Flexibility design problems are a class of problems that appear in strategic decision-making across industries, where the objective is to design a ($e.g.$, manufacturing) network that affords flexibility and adaptivity. The underlying combinatorial nature and stochastic objectives make flexibility design problems challenging for standard optimization methods. In this paper, we develop a reinforcement learning (RL) framework for flexibility design problems. Specifically, we carefully design mechanisms with noisy exploration and variance reduction to ensure empirical success and show the unique …
A Large Language Model-based multi-agent manufacturing system for intelligent shopfloor
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Key Laboratory of System Control and Information Processing, Ministry of Education of China（系统控制与信息处理教育部重点实验室）；Yangzhou University（扬州大学）
Flexible Manufacturing Systems Intralogistics: Dynamic Optimization of AGVs and Tool Sharing Using Coloured-Timed Petri Nets and Actor-Critic RL with Actions Masking
South Westphalia University of Applied Sciences（西南莱茵河应用科学大学）
Logic-informed reinforcement learning for cross-domain optimization of large-scale cyber-physical systems
State Key Laboratory of Robotics and Intelligent Systems（机器人与智能系统国家重点实验室）；Shenyang Institute of Automation（沈阳自动化研究所）；Chinese Academy of Sciences（中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
Constrained Reinforcement Learning for Dynamic Material Handling
Research Institute of Trustworthy Autonomous Systems (RITAS), Southern University of Science and Technology（南方科技大学可信自主系统研究院）；Southern University of Science and Technology（南方科技大学）；Trustworthiness Theory Research Center, Huawei Technologies Co., Ltd（华为技术有限公司可信理论研究中心）
Purpose of review: Recent advances in sensing, actuation, and computation have opened the door to multi-robot systems consisting of hundreds/thousands of robots, with promising applications to automated manufacturing, disaster relief, harvesting, last-mile delivery, port/airport operations, or search and rescue. The community has leveraged model-free multi-agent reinforcement learning (MARL) to devise efficient, scalable controllers for multi-robot systems (MRS). This review aims to provide an analysis of the state-of-the-art in distributed MARL for multi-robot cooperation. Recent findings: De…
Reinforcement Learning (RL) is emerging as tool for tackling complex control and decision-making problems. However, in high-risk environments such as healthcare, manufacturing, automotive or aerospace, it is often challenging to bridge the gap between an apparently optimal policy learnt by an agent and its real-world deployment, due to the uncertainties and risk associated with it. Broadly speaking RL agents face two kinds of uncertainty, 1. aleatoric uncertainty, which reflects randomness or noise in the dynamics of the world, and 2. epistemic uncertainty, which reflects the bounded knowledge…
Northeastern University（东北大学）；University of Waterloo（滑铁卢大学）
Biopharmaceutical manufacturing faces critical challenges, including complexity, high variability, lengthy lead time, and limited historical data and knowledge of the underlying system stochastic process. To address these challenges, we propose a green simulation assisted model-based reinforcement learning to support process online learning and guide dynamic decision making. Basically, the process model risk is quantified by the posterior distribution. At any given policy, we predict the expected system response with prediction risk accounting for both inherent stochastic uncertainty and model…
A Sliding-Window-Based Reinforcement Learning for Dynamic Assembly Flow Shop Scheduling with Multi-Product Delivery
School of Electromechanical Engineering, Guangdong University of Technology（广东工业大学机电工程学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
Intelligent Collaborative Optimization for Rubber Tyre Film Production Based on Multi-path Differentiated Clipping Proximal Policy Optimization
State Key Laboratory of Physical Oceanography and the Faculty of Information Science and Engineering, Ocean University of China（物理海洋学国家重点实验室和信息科学与工程学院，中国海洋大学）；School of Mathematical and Computer Sciences, Heriot-Watt University（数学与计算机科学学院，赫瑞-沃德大学）；Mesnac Co., Ltd.（梅斯纳克公司）
Industrial robots are widely used in various manufacturing environments due to their efficiency in doing repetitive tasks such as assembly or welding. A common problem for these applications is to reach a destination without colliding with obstacles or other robot arms. Commonly used sampling-based path planning approaches such as RRT require long computation times, especially in complex environments. Furthermore, the environment in which they are employed needs to be known beforehand. When utilizing the approaches in new environments, a tedious engineering effort in setting hyperparameters ne…
Multi-Mode Process Control Using Multi-Task Inverse Reinforcement Learning
State Key Laboratory of Industrial Control Technology, Institute of Cyber-Systems and Control, Zhejiang University（工业控制技术国家重点实验室，系统与控制研究院，浙江大学）；Department of Chemical Engineering, Chung-Yuan Christian University（化学工程系， Chung-Yuan Christian University）；Department of Chemical and Materials Engineering, University of Alberta（化学与材料工程系，阿尔伯塔大学）
Optimizing Job Shop Scheduling in the Furniture Industry: A Reinforcement Learning Approach Considering Machine Setup, Batch Variability, and Intralogistics
Faculty of Wood Technology and Construction（木材技术与建筑学院）；Rosenheim Technical University of Applied Sciences（罗森heim应用技术大学）；Center for Research, Development and Transfer（研究、开发与转让中心）；Faculty of Management and Engineering（管理与工程学院）
Forschungszentrum Jülich（于利希研究中心）；RWTH Aachen University（亚琛工业大学）；Technische Universität Berlin（柏林工业大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Korea University（高丽大学）
The ability to handle single molecules as effectively as macroscopic building-blocks would enable the construction of complex supramolecular structures inaccessible to self-assembly. The fundamental challenges obstructing this goal are the uncontrolled variability and poor observability of atomic-scale conformations. Here, we present a strategy to work around both obstacles, and demonstrate autonomous robotic nanofabrication by manipulating single molecules. Our approach employs reinforcement learning (RL), which finds solution strategies even in the face of large uncertainty and sparse feedba…
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；School of Intelligent Systems Science and Engineering, Jinan University（暨南大学智能系统科学与工程学院）
The dynamic job-shop scheduling problem (DJSP) is a class of scheduling tasks that specifically consider the inherent uncertainties such as changing order requirements and possible machine breakdown in realistic smart manufacturing settings. Since traditional methods cannot dynamically generate effective scheduling strategies in face of the disturbance of environments, we formulate the DJSP as a Markov decision process (MDP) to be tackled by reinforcement learning (RL). For this purpose, we propose a flexible hybrid framework that takes disjunctive graphs as states and a set of general dispatc…
ABV-Indian Institute of Information Technology and Management, Gwalior（ABV印度信息技术与管理学院（瓜廖尔））
Robotics has proved to be an indispensable tool in many industrial as well as social applications, such as warehouse automation, manufacturing, disaster robotics, etc. In most of these scenarios, damage to the agent while accomplishing mission-critical tasks can result in failure. To enable robotic adaptation in such situations, the agent needs to adopt policies which are robust to a diverse set of damages and must do so with minimum computational complexity. We thus propose a damage aware control architecture which diagnoses the damage prior to gait selection while also incorporating domain r…
Dongeui University（东义大学）；Pusan National University（釜山国立大学）
Process mining is a methodology for the derivation and analysis of process models based on the event log. When process mining is employed to analyze business processes, the process discovery step, the conformance checking step, and the enhancements step are repeated. If a user wants to analyze a process from multiple perspectives (such as activity perspectives, originator perspectives, and time perspectives), the above procedure, inconveniently, has to be repeated over and over again. Although past studies involving process mining have applied detailed stepwise methodologies, no attempt has be…
Hitachi America Ltd.（日立美国有限公司）；Stevens Institute of Technology（史蒂文斯理工学院）
Dynamic dispatching is one of the core problems for operation optimization in traditional industries such as mining, as it is about how to smartly allocate the right resources to the right place at the right time. Conventionally, the industry relies on heuristics or even human intuitions which are often short-sighted and sub-optimal solutions. Leveraging the power of AI and Internet of Things (IoT), data-driven automation is reshaping this area. However, facing its own challenges such as large-scale and heterogenous trucks running in a highly dynamic environment, it can barely adopt methods de…
Institute for Applied Materials - Computational Materials Science (IAM - CMS), Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院应用材料研究所——计算材料科学方向（IAM-CMS））；Intelligent Systems Research Group (ISRG), Karlsruhe University of Applied Sciences（卡尔斯鲁厄应用技术大学智能系统研究组（ISRG））
In real-world decision optimization, often multiple competing objectives must be taken into account. Following classical reinforcement learning, these objectives have to be combined into a single reward function. In contrast, multi-objective reinforcement learning (MORL) methods learn from vectors of per-objective rewards instead. In the case of multi-policy MORL, sets of decision policies for various preferences regarding the conflicting objectives are optimized. This is especially important when target preferences are not known during training or when preferences change dynamically during ap…
Arizona State University（亚利桑那州立大学）；Facebook AI Research（脸书人工智能研究院）
Humans and animals are capable of quickly learning new behaviours to solve new tasks. Yet, we often forget that they also rely on a highly specialized morphology that co-adapted with motor control throughout thousands of years. Although compelling, the idea of co-adapting morphology and behaviours in robots is often unfeasible because of the long manufacturing times, and the need to re-design an appropriate controller for each morphology. In this paper, we propose a novel approach to automatically and efficiently co-adapt a robot morphology and its controller. Our approach is based on recent a…
Facilitating Reinforcement Learning for Process Control Using Transfer Learning: Overview and Perspectives
College of Control Science and Engineering Zhejiang University（控制科学与工程学院浙江大学）；Department of Chemical Engineering Chung-Yuan Christian University（化学工程系 Chung-Yuan Christian University）
