The phase unwrapping (PU) for Permanent Scatterers (PS) is a key step to obtaining accurate urban mapping results (e.g., a height of a high-rise building) in the time-series InSAR workflow. The phase continuity assumption restricts the existing single-baseline (SB) PU algorithm, so it cannot obtain accurate results in an urban area with high-rise buildings. At the same time, the multi-baseline (MB) method has poor noise robustness for no use of global interferometric phase information. Here, an MB PU algorithm is proposed based on the two-stage programming approach (TSPA) and the general SB PU workflow for the sparse permanent scatterers. Then, we implemented the proposed method into the PSInSAR procedure and estimated building heights in urban areas of southern Chengdu, China. Satisfactory results were obtained for buildings with various heights as assessed by the Google Earth ® image and in situ height measurements. Thus, the developed method is valid and effective in building height retrievals in urban areas.