We propose a compact speckle spectrometer that utilizes micro-nanostructures processed by femtosecond lasers on sapphire surfaces as scattering media. The spectral resolution is 0.5 nm, and the entire system is compact and stable. At the same time, the convolutional long short-term memory network (CNN-LSTM) was introduced into the denoising algorithm. Compared with traditional reconstruction algorithms, this method not only ensures rapid spectral reconstruction but also offers better reconstruction accuracy. It can effectively reduce the reconstruction error caused by the reduction of speckle autocorrelation caused by environmental noise and prolong the stability time of the system.