With the prevalence of social media platforms, rumors have been a serious social problem. Notably, existing rumor detection methods simply provide detection labels while ignoring their explanation. However, illustrating the reasons why a suspicious statement is a rumor is essential. To address this realistic scenario, we propose a novel Evidence-Providing Rumor Detection model called EPRD. EPRD incorporates a wide variety of information from both prior knowledge sources and current comments. It also learns bilaterally friendly representations for interpretable rumor detection. EPRD first retrieves evidence from prior knowledge sources and checks the relationship between the given statement and its evidence. Our model then constructs two heterogeneous graph objects to simulate the propagation layout of the current comments and evidence relationships. Finally, EPRD integrates the GraphSAGE component and attention mechanism to detect rumors. To the best of our knowledge, we propose the first model that incorporates prior knowledge to verify rumors and boost credibility. Experiments on two real-world Twitter datasets demonstrate that EPRD consistently exhibits the best rumor detection performance. Moreover, EPRD outperforms other baselines in the early rumor detection task.