Incorporating knowledge graph as side information has become a new trend in recommendation systems. Recent studies regard items as entities of a knowledge graph and leverage graph neural networks to assist item encoding, yet by considering each relation type independently. However, relation types are often too many and sometimes one relation type involves too few entities. We argue that there may exist some latent relevance among relations in KG. It may not necessary nor effective to consider all relation types for item encoding. In this paper, we propose a VRKG4Rec model (Virtual Relational Knowledge Graphs for Recommendation), which clusters relations with latent relevance to generates virtual relations. Specifically, we first construct virtual relational graphs (VRKGs) by an unsupervised learning scheme. We also design a local weighted smoothing (LWS) mechanism for node encoding on VRKGs, which iteratively updates a node embedding only depending on the node itself and its neighbors, but involve no additional training parameters. LWS mechanism is also employed on a user-item bipartite graph for user representation learning, which utilizes item encodings with virtual relational knowledge to help train user representations. Experiment results on two public datasets validate that our VRKG4Rec model outperforms the state-of-the-art methods. The implementations are available at https://github.com/lulu0913/VRKG4Rec.