Time-Series Forecasting Through Contrastive Learning with a Two-Dimensional Self-attention Mechanism

Contrastive learning methods have impressive capabilities in time-series representation; however, challenges in capturing contextual consistency and extracting features that meet the requirements of representation learning remain. To address these problems, this study proposed a time-series prediction contrastive learning model based on a two-dimensional self-attention mechanism. The main innovations of this model were as follows: First, long short-term memory (LSTM) adaptive pruning was used to form two subsequences with overlapping parts to provide robust context representation for each timestamp. Second, the model extracted sequence data features in both global and local dimensions. In the channel dimension, the model encoded sequence data using a combination of a self-attention mechanism and dilated convolution to extract key features for capturing long-term trends and periodic changes in data. In the spatial dimension, the model adopted a sliding-window self-attention mechanism to encode sequence data, thereby improving its perceptual ability for local features. Finally, the model introduced a self-correlation attention mechanism that converted the similarity calculation from the real domain to the frequency domain through a Fourier transform, better capturing the periodicity and trends in the data. The experimental results showed that the proposed model outperformed existing models in multiple time-series prediction tasks, demonstrating its effectiveness and feasibility in time-series prediction tasks.