The novel semimetal tantalum phosphide (TaP) is known for its notable physical property including high carrier mobility, zero bandgap structure and superconductivity. However, there is no prior record of TaP as a saturable absorber (SA) within ultrafast fiber lasers. In this study, ultrasound assisted liquid phase exfoliation is applied to fabricate TaP quantum dots (QDs). Then QDs are subsequently deposited onto the tapered waist of specialized fiber to create TaP QDs-SA. The evaluation of nonlinear optical properties is conducted through two-armed balanced probing device, which yields noteworthy results including a modulation depth of 10.8 % and a saturation intensity of 0.52 KW/cm2. The TaP QDs-SA is applied in Er-doped fiber laser (EDFL) which demonstrates that a stable mode-locked pulse output with a repetition frequency of 21.01 MHz and a pulse width of 552 fs are obtained. The maximum achieved output power is 3.545 mW. The intrinsic semimetal properties of TaP QDs underscore its potential in the realm of ultrafast photonic and nonlinear optical devices.