We present a new ab initio method for electronic structure calculations of materials at finite temperature (FT) based on the all-electron quasiparticle self-consistent GW (QPscGW) approximation and Keldysh time-loop Green's function approach. We apply the method to Si, Ge, GaAs, InSb, and diamond and show that the band gaps of these materials universally decrease with temperature in contrast with the local density approximation (LDA) of density functional theory (DFT) where the band gaps universally increase. At temperatures of a few eV the difference between quasiparticle energies obtained in FT-QPscGW and FT-LDA approaches significantly reduces. This result suggests that existing simulations of very high temperature materials based on the FT-LDA are more justified then it might appear from well-known LDA band gap errors at zero-temperature.