Hardware Acceleration of Transformer Networks using FPGAs

Natural Language Processing (NLP) allows program computers to process and analyze large amounts of natural language data. In the last few years, NLP has shown tremendous growth, with many organizations presenting models such as BERT (Bidirectional Encoder Representations from Transformers), GPT2 (Generative Pre-trained Transformer 2), GPT3 (Generative Pre-trained Transformer 3), etc. The cornerstone of these models and the reason for the growth of the NLP is mainly due to the Transformer networks. However, very few architectures have been presented, for the acceleration of the Transformer networks using FPGAs. In this paper, we propose a novel architecture for Transformer Networks optimized on FPGAs. The performance evaluation on a Xilinx Alveo U200 FPGA achieved up to 80.5x speed-up over a single-core CPU and up to 2.3x speedup over a 40-thread Xeon CPU running BERT model.