Learning-Based Edge-Device Collaborative DNN Inference in IoVT Networks

Deep neural network (DNN) is a promising technology for Internet of Visual Things (IoVT) devices to extrct their visual information from unstructured data. However, it is hard to deploy a complete DNN model at resource-constrained IoVT devices to fulfill their latency, energy, and inference accuracy demands. Exploiting the reachable and available computing resources of IoVT devices and mobile-edge computing (MEC) servers, we propose an edge-device collaborative DNN inference framework to empower resource-constrained IoVT devices to perform DNN-based inference. Especially, the DNN model partition separates the DNN model into two parts, which are deployed on both the IoVT devices and multiaccess MEC server for performing inference collaboratively. The DNN early exit and computation resource allocation are employed to accelerate the DNN inference while guaranteeing the inference accuracy. Moreover, a metric to measure the inference performance of average latency and accuracy (IPLA) is designed. Joint multiuser DNN partitioning, early exit point selection, and computation resource allocation are optimized to maximize the tradeoff performance of inference latency and accuracy. We model the optimized problem as an Markov decision process and propose a deep deterministic policy gradient-based edge-device collaborative DNN inference algorithm to solve the problem of huge state space and high-dimensional continuous actions. Experiments are conducted with the Alexnet model on the data set of CIFAR-10 and Resnet-50 model on the data set of ImageNet. Simulation results verify that the proposed algorithm speeds up the overall inference execution of IoVT devices while guaranteeing inference accuracy.