Tumor heterogeneity plays a key role in prostate cancer prognosis, therapy selection, relapse, and acquisition of treatment resistance. Prostate cancer presents a heterogeneous diversity at inter- and intra-tumor and inter-patient levels which are influenced by multiple intrinsic and/or extrinsic factors. Recent studies have started to characterize the complexity of prostate tumors and these different tiers of heterogeneity. In this review, we discuss the most common factors that contribute to tumoral diversity. Moreover, we focus on the description of the in vitro and in vivo approaches, as well as high-throughput technologies, that help to model intra-tumoral diversity. Further understanding tumor heterogeneities and the challenges they present will guide enhanced patient risk stratification, aid the design of more precise therapies, and ultimately help beat this chameleon-like disease.