Faraday rotation (FR) measurements on three thin single crystalline samples of yttrium iron garnet doped with Sn, Zr, and Sb as a function of temperature in the near infrared region show a monotonic variation of the magneto-optical signal as the temperature is decreased from 300 to about 50 K. At this point the FR signal levels off. Moreover, the slope of the plot for the sample, doped with Sn, changes sign below this temperature, at particular wavelengths. An explanation of the observed phenomena is given in terms of the energy levels of the Fe2+ ions in the different sites of the crystal and the temperature dependence of their populations caused by the relative orientation between the local symmetry axis of the specific site and the direction of the sample magnetization. Hysteresis loops of the Faraday rotation as a function of applied magnetic field have been also measured showing the presence of a remanence of the sample magnetization.