Hyperspectral imaging (HSI) has been effectively used in the nondestructive assessment of food quality in recent years. However, the identification of moldy objects using HSIs still faces challenges, including slow detection speed and poor identification accuracy. To address these challenges, this study proposes a three-dimensional hyperspectral mold detection (3D-HMD) approach. The model utilizes multiple 3D convolution (3DMC) modules as the backbone network for optimizing spectral-spatial feature extraction and introduces an attention mechanism to promote the feature information of different hyperspectral bands. A feature pyramid network (FPN) is then used to fuse classification features outputted from the backbone network for feature enhancement. To improve the recognition efficiency for moldy targets, a detection head module derived from the field of object detection is introduced to achieve HIS object-level classification. The experimental results indicate that the detection speed of the proposed model is nearly tenfold greater than that of traditional algorithms, such as 1D-RNN and 3D-CNN, with a mean average precision (mAP) of 81.63 %. Overall, the 3D-HMD model demonstrates remarkable efficiency and accuracy in recognizing moldy peanuts, leading to suitable applications for food quality detection.