Recognition of catfish fillets using computer vision toward automated singulation

Abstract Catfish fillets are produced from whole catfish through a series of processing operations including de‐heading, gutting, filleting, skinning, trimming, and freezing. Catfish processing facilities usually chill or freeze a certain number of catfish fillets as backup storage. Later, catfish fillets are pulled out of storage for further processing, where significant manual labor is needed to separate and flatten all the fillets for downstream processing, such as portioning, breading, and individually quick‐frozen (IQF). Due to the labor force shortage and increasing labor cost, there is a pressing need to automate the singulation task, thereby reducing labor dependence. Machine vision technology has been researched for automated quality evaluation of fish products. However, research is lacking on using machine vision for automated singulation of fish fillets. This study presents a novel machine vision system consisting of a color camera for the recognition of the folding status and orientations of catfish fillets toward realizing automated fillet singulation. A set of 400 images of catfish fillets in four different orientations was captured and annotated for each catfish fillet. Two deep learning‐based image segmentation models, that is, YOLOv8 and SegFormer‐B5, were trained on the dataset for catfish fillet recognition. YOLOv8 outperformed SegFormer in catfish recognition and achieved overall masked mAP (mean average precision) scores of 97.1% and 97.4% for underwater and out‐of‐water catfish fillets, respectively. The vision system combined with YOLOv8 has the potential to automate the recognition and subsequent handling operations of catfish fillets. Practical applications The U.S. catfish industry is shrinking and facing great challenges because of international market competition and increasing production costs. The outbreak of COVID‐19 revealed a sharp deficiency in the labor force for the entire catfish industry, which has relied on manual labor for various processing operations. Mandatory lockdowns severely disrupted seafood supply chains and labor access. The tightening labor market has resulted in the delay of not only catfish processing but also production and distribution. It is imperative to increase the level of automation for processing operations such as IQF preparation process. At present, each processing facility needs 6–10 laborers for the manual catfish fillet IQF preparation process, implying that automating the process would potentially save hundreds of thousands of dollars in annual labor costs per processing line. The automation technology developed in this article has the potential to benefit the U.S. catfish processor by minimizing labor dependence and costs.