Fringe projection profilometry (FPP) has been more widely applied in fields such as intelligent manufacturing and medical plastic surgery. Recovering the three-dimensional (3D) surface of an object from a single frame image has always been the pursued goal in FPP. The color fringe projection method is one of the most potential technologies to realize single-shot 3D imaging because of the multi-channel multiplexing. Inspired by the recent success of deep learning technologies for phase analysis, we propose a novel single-shot 3D shape measurement approach named color deep learning profilometry (CDLP). Through `learning' on extensive data sets, the properly trained neural network can gradually `predict' the crosstalk-free high-quality absolute phase corresponding to the depth information of the object directly from a color fringe image. Experimental results demonstrate that our method can obtain accurate phase information acquisition and robust phase unwrapping without any complex pre/post-processing.