Deep Reinforcement Learning aided No-wait Flow Scheduling in Time-Sensitive Networks

Emerging latency-sensitive applications (e.g., industrial control, in-vehicle networks) require that the networks guaranteed data delivery with low, bounded latency. To meet this requirement, the IEEE 802.1 Working Group developed the time-sensitive networks (TSN) standard to enable deterministic communication on standard Ethernet. TSN technology is developed to enable deterministic communication using traffic scheduling and shaping technology. However, while the TSN standards define the mechanisms to handle scheduled traffic, it does not specify algorithms to compute fine-grained traffic scheduling policy. Current TSN flow scheduling schemes largely rely on a manual process, requiring knowledge of the traffic pattern and network topology features. Inspired by recent successes in applying reinforcement learning in online control, we propose a deep reinforcement learning aided no-waiting flow scheduling algorithm in TSN. Extensive simulations are performed to verify that our algorithm can find the optimal solution in an acceptable time.