StereoYolo+DeepSORT: a framework to track fish from underwater stereo camera in situ

This paper presents a 3D multiple object detection and tracking framework for identifying and quantifying changes in fish behaviour through tracking the 3D position, distance and speed of fish with respect to an underwater stereo camera. The framework consists of six essential modules based on 3D object detection to identify fish and multiple object tracking algorithms to track the fish in sequential frames. In particular, the latest version of Yolo (Yolov7) is utilised for object detection and the deep SORT algorithm is used for multiple object tracking. The framework was tested using videos captured from an underwater stereo camera in an industrial-scale sea-based fish farm. The results showed that the framework was able to accurately detect and track multiple fish in 3D. The fish position, distance and speed relative to the camera were also successfully detected. The results of this study demonstrate the effectiveness of this framework in identifying and quantifying changes in fish behaviour. The proposed novel framework has the potential to greatly enhance our understanding of fish behaviour in their natural habitats, leading to new insights into fish ecology and behaviour, while at the same time, it can enable researchers to study fish behaviour in a more detailed and accurate way.