Spatiotemporal Fusion Transformer for large-scale traffic forecasting

The way humans travel and even their daily commute, is gradually expanding beyond the confines of counties and cities. Traffic between counties, cities, and even across the entire state is increasingly becoming a common aspect of daily activities. The demand for traffic flow forecasting covering larger geographical areas and longer time spans is ongoing. However, existing studies lack targeted deep model proposals for large-scale forecasting. To address this gap, we propose Spatiotemporal Fusion Transformer (STFT). Specifically, we propose three modules on top of the Transformer architecture: (i) Seasonality Encoding, based on the multi-periodicity inherent in traffic flow to facilitate the extraction of more predictable time-variant components from complex patterns. (ii) Tubelet Embedding, partitioning the input into Tubelets as input tokens for the Transformer. The Tubelet design not only achieves quadratic reductions in computational and memory usage but also enhances spatiotemporal locality feature modelling. (iii) Token Permutator, leveraging diffusion graph to model the spatiotemporal dynamics as a token permutation process. The graph representation is then projected by a proposed Hadamard Mapper to circumvent the anomaly sensitivities of Graph Neural Networks in large-scale computations. Experimental results on five real-world datasets indicate that STFT can cater to collaborative forecasting at diverse scales (subdivision, county, municipal, state) that not only outperforms state-of-the-art methods but also enjoys a large speedup of up to 4.46×. Lastly, we also find that compared to independent forecasting for each subregion, large-scale collaborative forecasting with STFT offers both better feature utilization and requires less computational cost.