Extracting Key Information from Shopping Receipts by Using Bayesian Deep Learning via Multi-modal Features

This research presents a new key information extraction algorithm from shopping receipts. Specifically, we train semantic, visual and structural features through three deep learning methods, respectively, and formulate rule features according to the characteristics of shopping receipts. Then we propose a multi-class text classification algorithm based on multi-modal features using Bayesian deep learning. After post-processing the output of the classification algorithm, the key information we seek for can be obtained. Our algorithm was trained on a self-labeled Chinese shopping receipt dataset and compared with several baseline methods. Extensive experimental results demonstrate that the proposed method achieves optimal results on our Chinese receipt dataset.