EdAR: An Experience-Driven Multipath Scheduler for Seamless Handoff in Mobile Networks

Multipath TCP (MPTCP) improves the bandwidth utilization in wireless network scenarios, since it can simultaneously utilize multiple interfaces for data transmission. However, with the fast growth of mobile devices and applications, link interruptions caused by handoffs still lead to drastic performance degradation in such scenarios. Typically, a series of packet losses on part of the links will block the transmission of the entire connection when handoff occurs. This paper proposes an Experience-driven Adaptive Redundant packet scheduler (EdAR) for MPTCP, aiming at achieving seamless handoffs in mobile networks. EdAR enables flexibly scheduling redundant packets with an experience-driven learning-based approach in the face of drastic network environment changes for multipath performance enhancement. To enable accurate learning and prediction, both the network environment and the best course of actions are jointly learned via a Deep Reinforcement Learning (DRL) agent, which we design with a hybrid structure to deal with the complexity of system states. Furthermore, both offline and online learning are utilized to allow the agent to adapt to different and changing network environments. Evaluation results show that EdAR outperforms the state-of-the-art MPTCP schedulers in most network scenarios. Specifically in mobile networks with frequent handoffs, EdAR brings $2\times $ improvement in terms of the overall goodput.