Carbon nanotube (CNTs) networks embedded in polymer matrix have been extensively studied over the recent years as a flexible thermoelectric (TE) transport medium. However, their power factor has been largely limited by the relatively inefficient tunneling transport at junctions between CNTs and the low density of conducting channels through the networks. In this work, we demonstrate that significant enhancement of power factor is possible by adding electrically insulating microscale particles in CNT networks. When silica particles of a few micrometer diameters were co-embedded in single-walled CNT-polydimethylsiloxane (PDMS) composites, both the electrical conductivity and the Seebeck coefficient were simultaneously enhanced, thereby boosting the power factor by more than a factor of six. We find that the silica microparticles excluded a large volume of the composite from the access of CNTs and caused CNT networks to form around them using the polymer as a binder, which in turn resulted in improved connectivity and alignment of CNTs. Our theoretical calculations based on junction tunneling transport show that the large power factor enhancement can be attributed to the enhanced tunneling with reduced junction distance between CNTs and the increased geometric factor due to better CNT alignment. Additional enhancement of power factor by more than a factor of two was achieved by sample compression due to the further improvement of CNT alignment.