PFOA is a representative perfluorinated compound that is used as a surfactant in various industrial fields. However, because PFOA has severe side effects due to its strong toxicity, such as carcinogenesis, liver damage, and immune system damage, it is crucial to enable PFOA detection with high sensitivity. Herein, we developed a perfluorooctanoic acid (PFOA) surface-enhanced Raman scattering (SERS) sensor using self-assembled p-phenylenediamine (SAp-PD) nanoparticles and an Ag SERS substrate. For the ultra-sensitive detection of PFOA, we synthesized and optimized SAp-PD, which shows a decrease in SERS intensities when reacting with PFOA. Using the Ag nanograss SERS substrate, the change in intensity that resulted from the SAp-PD and PFOA reaction was amplified. Consequently, we detected the 1.28 pM (detection limit) of PFOA in distilled water. Moreover, PFOA molecules were successfully detected in samples of the PFOA-coated frying pan and rice extraction at concentrations up to 1.69 nM and 10.3 μM, respectively.