An improved feature-time Transformer encoder-Bi-LSTM for short-term forecasting of user-level integrated energy loads

Accurate prediction of energy demand is crucial to guarantee the secure and stable functioning of integrated energy systems. User-level integrated energy loads are characterized by substantial fluctuations, high randomness, and intricate coupling relationships across multiple energy sources. In this study, an improved Feature-Time Transformer Encoder-Bi-LSTM (FTTrans-E-BL) model, based on the Transformer architecture, is proposed for short-term forecasting of user-level integrated energy loads. Firstly, Windowed Time-Lagged Cross-Correlation (WTLCC) is employed to conduct correlation analysis of influential factors, thereby identifying the factors with strong correlation and their corresponding time lags. Then, an innovative encoder-decoder architecture is proposed for predicting integrated energy loads. In the encoder stage, a feature attention mechanism is employed to acquire the coupling relationships between the integrated energy loads, while a time attention mechanism is used to investigate the significance of the hidden states in every time step. In the decoder stage, bidirectional LSTM is adopted to capture time-series features. Next, Pareto optimization is applied to update the weights of losses to balance multiple forecasting tasks. Finally, on the basis of the aforementioned prediction model, an extension of the probabilistic forecasting model is created. Experiments are carried out to verify the efficacy of the suggested model on real data. In comparison to the traditional LSTM model, the evaluation metrics (e.g. RMSE) for the proposed model’s prediction of electricity, cooling, and heating load improve by 87.467%, 78.300%, and 81.512%, respectively. The experimental findings demonstrate the advantages of the presented prediction model.