A comprehensive framework for explainable cluster analysis

Machine learning has proven to be a powerful tool for knowledge extraction from large data sets across different domains. Data quality and results interpretability are essential when applying machine learning to inform decision-making processes. This is especially true for clustering methods, which are frequently employed for extracting knowledge from large data sets, due to their unsupervised nature. Although there are significant recent developments in explainable artificial intelligence (XAI) applied to unsupervised problems, they focus primarily on cluster interpretability and often overlook data quality challenges. Moreover, these developments are typically designed to use specific clustering algorithms, limiting their adaptability to incorporate alternative techniques. We propose a novel and comprehensive four-step sequential framework for explainable cluster analysis on high-dimensional mixed-type data to address these limitations. The framework encompasses data preprocessing, dimensionality reduction, clustering, and classification to ensure robust and explainable results. The proposed methodology has also been implemented in an open-source Python package called Clust-learn, designed to be accessible and customizable for researchers and practitioners. The framework has been validated by applying a case study focusing on large-scale assessments in education, effectively illustrating the strength and usefulness of the methodology in extracting and synthesizing knowledge from complex real-world data.