Raman spectra, supplemented by powder x-ray diffraction and magnetization data of isovalently $A$- and $B$-site substituted ${\mathrm{BiFeO}}_{3}$ in the ${\mathrm{Bi}}_{1\ensuremath{-}x}{\mathrm{La}}_{x}{\mathrm{FeO}}_{3}$ ($0\ensuremath{\le}x\ensuremath{\le}1$), ${\mathrm{Bi}}_{1\ensuremath{-}x}{\mathrm{Tb}}_{x}{\mathrm{FeO}}_{3}$ ($0\ensuremath{\le}x\ensuremath{\le}0.2$), and ${\mathrm{Bi}}_{0.9}{\mathrm{Sm}}_{0.1}{\mathrm{Fe}}_{1\ensuremath{-}x}{\mathrm{Mn}}_{x}{\mathrm{O}}_{3}$ ($0\ensuremath{\le}x\ensuremath{\le}0.3$) series, are presented. A good agreement between the structural transitions observed by x-ray diffraction and the vibrational modes observed in the Raman spectra is found over the whole substitutional ranges, and in particular we find spectroscopic signatures of a $\mathrm{Pb}{\mathrm{ZrO}}_{3}$-type structure for ${\mathrm{Bi}}_{0.8}{\mathrm{La}}_{0.2}{\mathrm{FeO}}_{3}$. Mode assignments in the substituted materials are made based on Raman spectra of the end-members ${\mathrm{BiFeO}}_{3}$ and $\mathrm{La}{\mathrm{FeO}}_{3}$. Moreover, by comparing spectra from all samples with $R3c$ structure, the phonon assignment in ${\mathrm{BiFeO}}_{3}$ is revisited. A close connection between the degree of octahedral tilt and the Raman shift of the ${A}_{1}$ oxygen ${a}^{\ensuremath{-}}{a}^{\ensuremath{-}}{a}^{\ensuremath{-}}$ tilt mode is established. An explanation for the strong second-order scattering observed in ${\mathrm{Bi}}_{1\ensuremath{-}x}{\mathrm{La}}_{x}{\mathrm{FeO}}_{3}$ and ${\mathrm{Bi}}_{1\ensuremath{-}x}{\mathrm{Tb}}_{x}{\mathrm{FeO}}_{3}$ is suggested, including the assignment of the previously mysterious ${\mathrm{BiFeO}}_{3}$ mode at 620 ${\mathrm{cm}}^{\ensuremath{-}1}$. Finally, the magnetization data indicates a transition from a cycloidal modulated state towards a canted antiferromagnet with increasing $A$-site substitution, while ${\mathrm{Bi}}_{0.9}{\mathrm{Sm}}_{0.1}{\mathrm{Fe}}_{1\ensuremath{-}x}{\mathrm{Mn}}_{x}{\mathrm{O}}_{3}$ with $x=0$ and $0.15$ exhibit an anomalous closing of the hysteresis loop at low temperatures. For low $A$-site substitution levels ($x\ensuremath{\le}0.1$) the decreasing Raman intensity of the Fe derived modes correlates with the partial destruction of the spin cycloid as the substitution level increases.