Carbon nanotubes (CNTs)-based polymer composites are great potential as flexible piezoresistive sensors to apply to wearable devices, electronic skins, etc. We herein prepared a type of polymer composites with a segregated structure derived from double-percolating structured polymer composites of two elastomers and CNTs. Expressly, CNTs were composited with two elastomers by melt blending, forming double-percolated structures. The composites with segregated structures were attained after removing the elastomeric phase where CNTs selectively located by solvent extraction, followed by hot compression to eliminate voids. This kind of segregated structure could reduce the percolation threshold of CNTs while maintaining superior stretchability of the elastomeric matrix. Compared to polymer composites with randomly distributed CNTs, the composites with the segregated structure posed nonrandom distribution of CNTs, forming a denser conductive network with the same content. Therefore, the composites exhibited different mechanoelectrical responses from the traditional CNTs-based polymer composites. The composites with the segregated structure showed a higher strain where the relative change of resistance increased significantly. Notably, the composites exhibited a linear (instead of the commonly nonlinear) piezoresistive response with improved sensitivity under low mechanical strains. The practical applications of the as-prepared composites as piezoresistive sensors showed reliable performance in detecting gentle touching, dropping impact, and human body motions.