Comparison for thermal imager performance assessment: TOD classifier versus YOLO-based models for object detection

Models for triangle orientation discrimination (TOD) have been proposed for performance evaluation of thermal imaging devices. For thermal imager assessment, human visual systems for TOD have been modeled and rigorously validated for a wide variety of image distortions through observer studies. As the conduct of observer trials is time-consuming and costly, also AI-based TOD models for imager assessment have been presented. Recently, camera systems with embedded automatic target recognition (ATR) are becoming increasingly important. So far it is an open question if the simple TOD task, as a classification problem with 4 classes, is suitable for providing similar evaluations and rankings for these thermal imaging devices as algorithms for more complex and slower tasks like object detection, e.g. for ATR. A widely used framework for object detection is "You Only Look Once" (YOLO).

In this work, performance assessments for TOD models and YOLO-based models are compared. Known image databases as well as synthetic images with triangles and natural backgrounds are degraded according to a unified device description with blur and image noise. The blur caused by optical diffraction and detector footprint is varied by multiple aperture diameters and detector sizes through the application of modulation transfer functions, while the image noise is varied by multiple noise error levels as Gaussian sensor noise. The TOD models are evaluated for the degraded images with triangles, while the YOLO models are applied to the degraded variants of the image databases. For different degradation parameters, the model precisions of the TOD models are compared to figures of merit of the YOLO models such as the mean average precision (mAP). Statistical uncertainties of the performance ranking for different degradation parameters of cameras and both TOD and YOLO models are investigated.