Order-by-disorder and spin-orbital liquids in a distorted Heisenberg-Kitaev model

The microscopic modeling of spin-orbit entangled $j=1/2$ Mott insulators such as the layered hexagonal iridates ${\mathrm{Na}}_{2}{\mathrm{IrO}}_{3}$ and ${\mathrm{Li}}_{2}{\mathrm{IrO}}_{3}$ has spurred an interest in the physics of Heisenberg-Kitaev models. Here we explore the effect of lattice distortions on the formation of the collective spin-orbital states that include not only conventionally ordered phases but also gapped and gapless spin-orbital liquids. In particular, we demonstrate that in the presence of distortions, i.e., spatial anisotropies of the exchange couplings, conventionally ordered states are formed through an order-by-disorder selection, which is not only sensitive to the type of exchange anisotropy but also to the relative strength of the Heisenberg and Kitaev couplings. The spin-orbital liquid phases of the Kitaev limit---a gapless phase in the vicinity of spatially isotropic couplings and a gapped ${Z}_{2}$ phase for a dominant spatial anisotropy of the exchange couplings---show vastly different sensitivities to the inclusion of a Heisenberg exchange. While the gapless phase is remarkably stable, the gapped ${Z}_{2}$ phase quickly breaks down in what might be a rather unconventional phase transition driven by the simultaneous condensation of its elementary excitations.