In computer vision field, image classification as a basic image processing task has been widely concerned. This paper puts forward an efficient attention network with dilated convolution named Dilated-CBAM for image classification. The dilated convolution is applied in the backbone of the residual network to extract the residual edge path features and integrate the global information of the processed image. The amount of network parameters is greatly reduced while the receptive field is expanded, and the network parameters are learnable. By embedding our spatial attention mechanism and channel attention mechanism, the model uses FCN to strengthen the effective information in the image, weaken the invalid information, and summarize the local features of the processed image. Combining the global information and local information, the time and space for network training are saved, while the effective image features can be extracted better. In the design of attention module, this work innovatively applies residual path in attention module for combining context information inside attention mechanism without adding parameters, and helps attention module extract features of interest in image classification task more accurately. In image classification, experiment, we verify the classification effect of the Dilated-CBAM model on Cifar-10 dataset, which is 2.5% higher than ResNet-18, and reaches the classification accuracy of 93.5%. In terms of the efficiency of network training, the Dilated-CBAM reduces the number of training epochs to about 10 on the basis of CBAM model, shortens the training time to about half of the original, which greatly testifies the training efficiency.