Polymer-ceramic composites combine the benefits of polymers and ceramics. In particular, the infiltration of the ceramic cathode with a Li-ion-conducting polymer in an all-solid-state Li battery enhances the utilization of the cathode active material (CAM) and enables the application of thicker cathodes with higher storage capacity. This concept has already been validated in our earlier work, in which a porous LiCoO2–Li6.45Al0.05La3Zr1.6Ta0.4O12 (LLZO:Al:Ta) composite cathode was fabricated by spark plasma sintering (SPS) technique. However, its performance stability was low. In the present work, the concept is modified using an LFP-LATP cathode with LiFePO4 as the CAM, Li1.5Al0.5Ti1.5(PO4)3 as the ion-conducting phase, and tape-casting with free sintering instead of SPS. Both tape-casting and free sintering are more relevant for large-scale production. The sintered LFP-LATP cathode is infiltrated with the MEEP polymer and LiC2NO4F6S2 ion-conducting salt. A full cell with the polymer-infiltrated cathode, LLZO:Al:Ta separator, and Li anode shows nearly full LFP utilization in the 100 μm thick cathode with an excellent area-specific storage capacity of above 3 mAh cm−2. However, after a few dozen cycles, a Li dendrite penetrates the separator leading to abrupt capacity fading. The prevention of Li dendrite formation remains a challenge for our future work.