Deep Multi-Representation Model for Click-Through Rate Prediction

Click-Through Rate prediction (CTR) is a crucial task for online advertising and recommender systems. Therefore, it has gained considerable attention in the past few years as it highly affects the revenue of several commercial platforms and online systems. The primary purpose of recent research emphasizes obtaining meaningful and powerful representations through mining low and high-feature interactions using various components such as Deep Neural Networks (DNN), CrossNets, or transformer blocks. However, models utilizing one representation for the input fields in each instance restrict the model's predictive power. Other models tend to be overly complicated to reach high input data expressiveness and predictive power. In this work, we propose a simple yet effective Deep Multi-Representation model (DeepMR) that is capable of learning informative representations by jointly training a mixture of two powerful feature representation learning components, namely DNNs and multi-head self-attentions. Furthermore, DeepMR integrates the novel residual with zero initialization (ReZero) connections to the DNN and the multi-head self-attention components for learning superior input representations. Experiments on three real-world datasets show that the proposed model significantly outperforms all state-of-the-art models with a relative improvement of up to 16.6% in the task of click-through rate prediction. Our implementation code and datasets are available here https://github.com/Shereen-Elsayed/DeepMR.