Surface enhanced Raman spectroscopy (SERS) is becoming an effective method of detecting organic molecules, and three-dimensional (3D) SERS substrates are promising in practical applications. To optimizing 3D SERS substrates, the extinction spectra and local electromagnetic fields of Ag-decorated ZnO nanorods (Ag-ZnO-NR) were studied by a finite-difference time-domain (FDTD) method. The diploe local surface plasmonic resonance (LSPR) was found to split into high- and low-frequency two branches, in which the low-frequency mode is predominant in contributing to the Raman enhancement, while the ZnO-NR radius and Ag-layer thickness play important roles in tuning the LSPR frequencies. Both of high- and low-frequency LSPRs were enhanced with the increase in the Ag-ZnO-NR length while the intensity of local electromagnetic field depends on the Ag-ZnO-NR density. The FDTD results were discussed with a comparison to the experimental data obtained from flower-like Ag-ZnO-NR 3D SERS substrates. In addition, the extinction spectra of Ag nanotube and Ag-decorated TiO2 and SiO2 nanorods were calculated and the refractive index of dielectric materials was found to be capable of tuning the LSPR of 3D SERS substrates as well. The principle revealed in this work is helpful for design and optimization of 3D SERS substrates.