We study the polarization-dependent laser performance of a novel, to the best of our knowledge, "mixed" Tm,Ho:CaYGdAlO4 crystal in the continuous-wave (CW) and mode-locked regimes. Both in terms of the CW tunability range (261 nm) and the minimum pulse duration (50 fs at 2078 nm, spectral width of 95 nm) in the mode-locked regime, σ-polarization is superior. With extended inhomogeneous spectral broadening due to structural and compositional disorder, Tm,Ho:CaYGdAlO4 is promising for few-optical-cycle pulse generation around 2 µm.