This work explores the problem of the personalization of the autonomous driving experience, leveraging the existing advanced driver-assistance systems (ADAS) through a combination of Reinforcement Learning (RL) algorithms and federated learning (FL) techniques. The problem is placed in the context of the interconnected vehicles with processing capabilities, and we demonstrate how that type of vehicle using FL can minimize the time needed to train an RL-based personalization model, taking advantage of the collective knowledge of multiple drivers with similar profiles in the same network. To demonstrate the effectiveness of the proposed method, we conducted experiments in a driving simulation environment. The goal of the RL was to dynamically select proper driving modes for a driver during a route. The results show that our solution can better balance drivers' stress in various situations, and also reduce the overall time needed for a model to adapt to a driver. Overall, the results show that our approach is a promising solution for driving mode personalization in ADAS. To provide a holistic view of the challenges for FL in collaborative scenarios, we also discuss the risk of attacks from adversarial users on private and sensitive data that are used in this process.