To effectively improve system autonomy, increase fault-tolerant resilience, solve low payload capacity and short endurance time of unmanned surface vehicles (USVs), there's a trend to deploy multiple USVs as a formation fleet. The formation path planning algorithms are essential to generate optimal trajectories and provide practical collision avoidance maneuvers to efficiently navigate the USV fleet. To ensure the optimality, rationality and path continuity of the formation trajectories, this paper presents a novel deterministic algorithm named multiple sub-target artificial potential field (MTAPF) based on an improved APF. The MTAPF belongs to the local path planning algorithm, which refers to the global optimal path generated by an improved heuristic A* algorithm. and the optimal path is divided by this algorithm into multiple sub-target points to form sub-target point sequence. The MTAPF can greatly reduce the probability that USVs will fall into the local minimum and help USVs to get out of the local minimum by switching target points. As an underactuated system, the USV is restricted by various motion constraints, and the MTAPF is presented to make the generated path compliant with USV's dynamics and orientation restrictions. The proposed algorithm is validated on simulations and proven to work effectively in different environments.