The 6th generation of mobile networks (6G) offers many advantages, including fast transmission rates, low delays, and ultra-dense networks. These benefits can solve the communication problems in the Internet of Vehicles (IoV). However, auto-mobiles as IoV terminals generate Non-Independent Identically Distributed (Non-IID) data while driving, making it necessary to introduce distributed machine learning into the IoV as a network that integrates intelligent computing and vehicle networking. To address the Non-IID of local data and the heterogeneity of local models, we propose a heterogeneous Federated Learning scheme in this paper. Based on the distributed architecture (Terminal - Edge device - Cloud) in the IoV, we designed a privacy protection scheme using Secure Multi-Party Computation (SMPC). This ensures that the terminals participating in Federated Learning get accurate calculation results without revealing useful information, thus preserving the privacy of local datasets. The privacy and security of the IoV based on Federated Learning (FedVPS) not only protect the privacy of the terminals but also improve communication efficiency, enabling accurate and efficient distributed machine learning. The aggregation method of Federated Learning is a prototype-based scheme that utilizes the effective information stored in local datasets. In the BIT-Vehicle dataset, FedVPS is not only more robust but also has excellent prediction accuracy. Compared to FedAvg, FedVPS has advantages in communication efficiency and model prediction accuracy.