Deep clustering has been a popular research topic in recent years. Yet previous deep clustering works are mostly designed for vectorized data or image data, which lack the ability to simultaneously exploit the sample (or node) attributes and their topological structure information and thus are not suitable for attributed graph clustering. In view of this, this paper presents a novel deep attributed graph clustering with graph attention network (DAGAT) approach, which jointly leverages the deep neural network (DNN) and the graph attention network (GAT) with two levels of attention mechanisms, i.e., the representation-wise attention and the node-wise attention. In our DAGAT approach, four modules are specifically incorporated, namely, the feature extraction module (FEM) via the autoencoder, the graph attention network module (GATM) to learn the features with topological information, the representation fusion module (RFM) to adaptively fuse the representations learned by the DNN and the GAT, and the self-supervised learning module (SSLM) for learning clustering-friendly representations. Experimental results have shown that our DAGAT approach outperforms the state-of-the-art approaches on multiple real-world datasets. The code is available at https://github.com/zbw0329/DAGAT.