Small targets in UAV aerial images are large in proportion, densely distributed, and complex in background, which leads to the existing algorithms to be prone to false and missed detection in target detection. In this paper, an ACN-YOLO small target detection algorithm is proposed to solve the problem of false and missed detection in UAV aerial images by using the ACmix attention mechanism based on a mixture of self-attention and convolution and an improved loss function. The algorithm is reconstructed on the basis of YOLOv7 to significantly reduce the number of network parameters while retaining more small target feature information, and use a large size detection head to match the small target size to improve the detection accuracy. The ACmix attention mechanism, which is a mixture of self-attention and convolution, is added to the feature fusion network to reduce the interference of irrelevant information. Introduce NWD Loss in the calculation of regression loss to compensate for the sensitivity of CIOU Loss to small target location differences. Using the VisDrone dataset for validation, the parameter amount of ACN-YOLO is reduced by 75% compared with the YOLOv7 network, while the mAP0.5 is improved by 3.8%, demonstrating that the algorithm in this paper can be effectively applied to the UAV aerial photography target detection task.