In this paper, the structural features of the iron phosphate glasses with Al2O3 (2.5∼15 mol%) and B2O3 (5∼15 mol%) were investigated by using Fourier transform infrared spectra and Raman spectra. The results show that the glass network of the studied iron phosphate glasses with Al2O3/B2O3 mainly consists of Q1 phosphate units. Generally, with the increasing content of Al2O3 and B2O3, Q1 and Q0 phosphate units increase and Q2 units decrease for the glasses with Al2O3/B2O3 ≤10 mol%, otherwise, an adverse variation is detected. The structural network of the iron-phosphate glasses is enhanced when the substitution of Al2O3/B2O3 is not higher than 10 mol%, as proven by the changes of the thermal parameters. Moreover, the reasons on the structural variations depending on Al2O3/B2O3 are discussed in detail. The conclusions provide researchers with the roles of Al2O3 and B2O3 in structural features of iron phosphate glasses as potential hosts for specific nuclear wastes.