Heavy metals accumulating in the human body produce physiological toxicity by interfering with the transport of human proteins and enzymes. Heavy metals detection is significant for food safety assurance. This review focuses on recent advances of heavy metals detection of food and agricultural products by surface-enhanced Raman spectroscopy (SERS). The article covers the SERS basic principles and advances in heavy metals detection, including mercury, arsenic, cadmium, lead, chromium among others. Insights in the potential of combining chemometrics and multivariate analysis with SERS and the exploration of novel SERS substrate platforms from both macro and micro scale are discussed. Finally, future application of SERS in heavy metal detection are prospected. SERS is a powerful and promising technique offering the advantages of simple sampling, rapid data collection and non-invasiveness. The findings of this study can allow better understanding of the heavy metals’ occurrence and the possibility of its detection using SERS.