Multi-Head Self-Attention Transformer for Dogecoin Price Prediction

Cryptocurrency market has witnessed a boom during the global pandemic and has proven as a strong investment with a wide institutional adoption. A time-series forecasting solution will play a vital role in analyzing the fluctuation of the bitcoin and altcoin markets. Dogecoin is one such altcoin that is a low-price, high-risk investment option garnering considerable interest this year. The variation of the price trend of this altcoin is studied using the multi-head attention mechanism implemented in a transformer, where the attention heads attend to the tokens that are relevant to each current token based on varying short-term and long-term dependencies. In this paper, a multi-head attention-based transformer encoder-decoder model is applied on the hourly data of the Dogecoin price for its prediction over time. The performance of the model has been evaluated using a number of evaluation metrics including MAE and predictive R-squared value. The model trained over the Dogecoin hourly price variation gave an impressive accuracy of 98.46% and R-squared value of 0.8616 comparable with the existing state-of-the-art cryptocurrency price forecasting models.