The outbreak of aggregative diseases in the process of sea cucumber cultivation has brought huge economic losses to aquaculture farmers. It is of positive significance to realize intelligent detection of abnormal behavior to avoid the outbreak of aggregative diseases. Therefore, this paper researches the approaches of intelligent recognition and behavior tracking of sea cucumbers. Fusing the Coordinated Attention and Bi-directional Feature Pyramid Network, the DT-YOLOv5 intelligent recognition model is proposed to enhance the representation ability and feature extraction ability. A multi-object behavior tracking approach is presented based on the automatic frame-matching coordinates, which can track multiple objects and calculate the volumes of exercise. The experimental results show that the precision, recall and AP50:95 are 99.43%, 98.91% and 84.89%, respectively. This research provides a theoretical support for the detection of abnormal behavior of aquatic animals during intensive aquaculture and has potential practical application value for protecting the welfare of sea cucumbers and improving the intelligence level of aquaculture.