Using real-time manufacturing data to schedule a smart factory via reinforcement learning

• Design a cyber-physical architecture and scheduling mechanism for smart factory. • Establish a genetic-programming-based scheduling rule library. • Develop a module to realize the dimension reduction and clustering of data. • Use RL to train the decision-making agent for appropriate rule selection. Under the background of intelligent manufacturing, internet of things and other information technologies have accumulated a large amount of data for manufacturing system. However, the traditional scheduling methods often ignore the production law and knowledge hidden in the manufacturing data. Therefore, this paper proposes a cyber-physical architecture and a communication protocol for smart factory, and a multiagent-system-based dynamic scheduling mechanism is given using contract net protocol. In the dynamic scheduling mechanism, the problem formulation module and scheduling point module are designed first. Then, a genetic programming (GP) method is proposed to form sixteen high-quality rules, which constitute the scheduling rule library. Meanwhile, combining with autoencoder, self-organizing mapping neural network and k-means clustering algorithm, the state clustering module is designed to realize the efficient clustering of production attribute vector. Moreover, an improved Q-learning algorithm is used to train the GP rule selector, so that the decision-making agent can choose the appropriate GP rule according to the production state at each scheduling point. Finally, the experimental results show that the proposed method has feasibility and superiority compared with other methods in real-time scheduling, and can effectively deal with disturbance events in the manufacturing process.