The knowledge map for most of the real world is incomplete, that is, there are problems of missing real facts and containing false facts. In recent years, most of the work, such as ConvE and TransE, reasoned the knowledge map by querying the implicit knowledge related to rules or based on the path, but the inference process was affected by large-scale long-distance and complex relationships, which led to the lack of interpretability and low training efficiency of the reasoning process. This paper proposed the optimization method of the reasoning process of the knowledge map based on the confrontation network, GAPO, The R-GCN auxiliary network is introduced into the GAN network to generate mixed data with high confidence as far as possible during the period of generating negative sample data, so as to improve the discriminator's ability to distinguish true and false triple facts. At the same time, reinforcement learning algorithm is introduced to treat the reasoning process of knowledge atlas as state space, and hierarchical information is used to ensure the reliability and authenticity of the reasoning link. The obtained hierarchical information data improves the interpretability of the reasoning process to a certain extent. The experiment shows that GAPO model has better performance than ConvE and TransE in reasoning, which proves that it is effective.