Solar drying can solve the problems of product quality caused by open sun drying and high energy consumption caused by conventional energy drying. However, due to the low solar heat collection efficiency of conventional flat-plate solar collectors, the maximum drying temperature of the existing solar drying system is typically <70 °C, making them difficult to be used in cold regions. Herein, we proposed a new solar drying system with high solar heat collection efficiency, high hot air outlet temperature, and high drying efficiency based on silica aerogels that are transparent to sunlight and have ultra-low thermal conductivity. The results show that the optimum thickness of the silica aerogel insulation layer is 10 mm. At ambient temperatures of −10 to 30 °C, the air outlet temperatures of the designed solar collector are 52–97 °C, increasing the air outlet temperature by 23 °C and 28 °C, respectively, compared to conventional solar collectors. The proposed solar drying system incorporating a 10-mm-thick silica aerogel insulation layer can meet the drying requirements of most agricultural products without an additional heat source and shorten the drying time by 58%.