Unsupervised malicious domain detection with less labeling effort

Since malware creates severe damage to the system, past studies leveraged various algorithms to detect malicious domains generated from Domain Generation Algorithms (DGAs). Although they achieved a promising performance, security practitioners had to acquire a large amount of fine-labeled dataset with a particular effort. Throughout the research, we propose a series of analysis to build a novel malicious domain detection method with the autoencoder in an unsupervised approach to overcome this limit. The contributions of our study are as follows. First, we proposed significant feature extraction methods that focused on the domain’s linguistic patterns and validated the proposed set of features effectively discriminate benign domains and malicious domains. Second, we established a malicious domain detection method with the autoencoder only with benign domains provided during the model training. Thus, we let a security practitioner build a malicious domain detection model with less labeling effort. Third, the proposed malicious domain detection model achieved a precise detection performance of 99% accuracy and F1 score. Lastly, our model maintains the aforementioned detection performance, although it is trained with a small training set; thus, the model reduces training dataset accumulation effort. Although our detection model cannot identify malicious domains’ origins, particular types of DGA, we evaluate security practitioners can easily implement a countermeasure against DGAs with less effort. In pursuit of precise malicious domain detection, we expect our study can be a concrete baseline for future works.