Affine Registration of Histological Images Using Transformer-Based Feature Matching

Fusing of information, obtained during the histological slides staining can be helpful for diagnosing and further patient treatment. However, when the slides are being prepared, tissues are subjected to deformations and registration is highly required. Automatic histological image registration is one of the most challenging parts of a histological tissues analysis. The situation is exacerbated by the lack of data and its diversity when the neural networks are susceptible to overfitting and low generalization ability. One of the core sub-problems of histological image registration is the calculation of the initial affine transformation before the final nonrigid registration. We propose a new method of affine registration that requires no histological data to learn and is based on knowledge transfer from nature domain. The results show that it outperforms existing methods by a large margin on most-commonly used histological images registration benchmark in terms of target registration error and produces less outliers. The code is available at https://github.com/VladPyatov/ImgRegWithTransformers .