Abstract Seafloor unconstrained environment video is shot in uncontrolled open sea area. There are multiple backgrounds, complex illumination and weather changes, and rapid growth of algae and attached lenses, which affect the stability of video shooting quality, resulting in difficulty in image recognition. At present, there is still no algorithm that is superior to other methods generally, and it is necessary to build a model combined with specific scenes and applications. In this paper, a fish identification method in tropical waters based on transfer learning under unconstrained environment is proposed. Firstly, the image is pre-processed by affine transformation to realize data enhancement. Furthermore, RestNet50 deep convolutional neural network is constructed based on transfer learning to compare the effect of fish recognition before and after transfer learning. The results show that, the accuracy and loss indicators are better than those of non-transfer learning when the trained model of imagenet is introduced as the initial weight of the network. When the model is trained to 150 epochs, the indicators begin to converge, which can better complete the fish identification task in tropical waters under unconstrained environment.