In this paper, we proposed an empirical playback-based deep reinforcement learning (DRL) approach to solve the problems related to autonomous path planning for unmanned aerial vehicles (UAVs), which is different from the traditional DRL-Deep Q Network (DQN). To improve the learning efficiency, we also proposed a step reward scheme to replace the traditional simple reward function and introduce it into the experience replay-based DRL-DQN approach in this paper to construct a new experience replay-based DRL-DQN. The results show that our scheme has a significant improvement in performance compared with the traditional scheme, and also makes a significant improvement in the adaptation of the UAV to the dynamic environment.