Mg, Fe double-doped LiTaO3 and LiNbO3 crystals have been grown by Czochralski method. The optical properties were measured by two-beam coupling experiments and transmitted facula distortion method. The results showed that the photorefractive response speed of Mg:Fe:LiTaO3 was about three times faster than that of Fe:LiTaO3, whereas the photo-damage resistance was two orders of magnitude higher than that of Fe:LiTaO3. In this paper, site occupation mechanism of impurities was also discussed to explain the high photo-damage resistance and fast response speed in Mg:Fe:LiTaO3 crystal.