The unitarily invariant probability measures on infinite Hermitian matrices have been classified by Pickrell, and by Olshanski and Vershik. This classification is equivalent to determining the boundary of a certain inhomogeneous Markov chain with given transition probabilities. This formulation of the problem makes sense for general $\beta$-ensembles when one takes as the transition probabilities the Dixon-Anderson conditional probability distribution. In this paper we determine the boundary of this Markov chain for any $\beta \in (0,\infty]$, also giving in this way a new proof of the classical $\beta=2$ case. Finally, as a by-product of our results we obtain alternative proofs of the almost sure convergence of the rescaled Hua-Pickrell and Laguerre $\beta$-ensembles to the general $\beta$ Hua-Pickrell and $\beta$ Bessel point processes respectively.