Deep reinforcement learning approach for dynamic capacity planning in decentralised regenerative medicine supply chains

AbstractDecentralized manufacturing has the benefits of fast fulfillment, reducing risks of distant delivery, and improving patient access to personalised regenerative medicine (PRM). Implementing the decentralised PRM manufacturing system successfully needs a capacity planning strategy involving inventory replenishment, capacity allocation, and demand sharing to mitigate the impacts of supplier disruption and satisfy demand with a high service level. However, existing methods for generating optimal capacity planning policies for such PRM systems require knowing the distributions of the supplier disruption and demand uncertainty, which is usually unknown for PRM supply chains. This study proposes a data-driven approach that can learn effective capacity planning policy under various manufacturing circumstances without knowing the exact distributions. The proposed approach utilises a production simulation model and a deep reinforcement learning method. Case study results demonstrate that the proposed method can outperform existing methods when ground-truth demand forecasts differ from priori estimations. The results also support that the proposed method not only can be applied in regenerative medicine but also in many other sectors.Keywords: Regenerative medicinedecentralised manufacturing systemreinforcement learningdynamic capacity planningsupply disruptiondemand uncertainty Disclosure statementNo potential conflict of interest was reported by the author(s).Data availability statementThe data that support the findings of this study are available from the corresponding author, C.-Y. Tseng, upon reasonable request.Additional informationFundingThe authors acknowledge that the research was supported by the BioFabUSA of Advanced Regenerative Manufacturing Institute [grant number T0171]. In addition, the simulation environment developed in this study is based on the concept depicted in work supported by the National Science Foundation [grant number EEC-1648035].Notes on contributorsChin-Yuan TsengChin-Yuan Tseng received his Ph.D. in Industrial Engineering with a specialisation in System Informatics and Control and a minor in Machine Learning from the Georgia Institute of Technology in 2023. His research focuses on simulation and AI for production systems and supply chain integration.Junxuan LiJunxuan Li is a senior scientist lead at Microsoft Business Emerging Technology, applying state-of-art Operation Research (OR) and Large Language Models (LLM) methodologies to business applications, e.g., ERP and CRM systems. Junxuan received his Ph.D. in Operations Research from Georgia Tech with a minor in AI, concentrating on sequential decision-making and dynamic control. The main application areas include smart supply chains, intelligent manufacturing, intelligent healthcare, field services, transportation, and e-commerce.Li-Hsiang LinLi-Hsiang Lin serves as an Assistant Professor in the Department of Mathematics and Statistics at Georgia State University. He earned his Ph.D. in Industrial Engineering with a specialisation in Statistics and a minor in Machine Learning from the Georgia Institute of Technology in 2020. His research focuses on various areas, including computer experiment modelling, nonparametric regression techniques, and the development of innovative methodologies for applications in engineering.Kan WangKan Wang is a Senior Research Engineer at the Georgia Tech Manufacturing Institute (GTMI) and leads the Advanced Manufacturing for BioEngineering Research (AMBER) laboratory. His research focuses on tissue engineering, biosensors, and biomanufacturing supply chain simulation. Dr. Wang earned his B.S. in Theoretical and Applied Mechanics from Peking University, M.S. in Aircraft Design from Beihang University, and Ph.D. in Industrial and Manufacturing Engineering from Florida State University. Since completing his Ph.D. degree in 2013, he has authored over 80 refereed journal papers and 4 book chapters. Dr. Wang's work continues to drive innovation at the intersection of advanced manufacturing and bioengineering.Chelsea C. White IIIChelsea C. White III Professor White holds the Schneider National Chair of Transportation and Logistics at Georgia Tech. His most recent research interests include analysing the role and value of real-time information for stress testing supply chains to improve next-generation manufacturing supply chain competitiveness, risk reduction, and for healthcare related supply chains, patient benefit. He is a Fellow of the IEEE, a Fellow of INFORMS, and an INFORMS Edelman Laureate. He is a former member of the board of directors of the Fortune 500 company Con-way, Inc. (NYSE: CNW, 2004–2015) and of the World Economic Forum Trade Facilitation Council.Ben WangBen Wang is a Professor Emeritus at the Georgia Institute of Technology (GT). He was the Executive Director of Georgia Tech Manufacturing Institute from 2012 to 2022. His professional focus is strengthening manufacturing competitiveness through technology, infrastructure, workforce, and policy. From 2017 to 2019, he served as Chair of the National Materials and Manufacturing Board of the National Academies of Sciences, Engineering, and Medicine. In addition to authoring or co-authoring more than 280 refereed journal papers, he co-authored three books. He is involved in two startups in 3D printing.