Comprehensive analysis of network robustness evaluation based on convolutional neural networks with spatial pyramid pooling

Connectivity robustness, crucial for network understanding, optimization, and repair, has been evaluated traditionally through time-consuming and often impractical simulations. Fortunately, machine learning provides a novel solution. However, unresolved challenges persist: performance in more general edge removal scenarios, capturing robustness via attack curves instead of directly training for robustness, scalability of predictive tasks, and transferability of predictive capabilities. Here, we try to address these challenges by designing a convolutional neural networks (CNN) model with spatial pyramid pooling networks (SPP-net), adapting existing evaluation metrics, redesigning the attack modes, introducing appropriate filtering rules, and incorporating the value of robustness as training data. Results indicate that the CNN framework consistently provides accurate evaluations of attack curves and robustness values across all removal scenarios when the evaluation task aligns with the trained network type. This effectiveness is observed for various network types, failure component types, and failure scenarios, highlighting the scalability in task scale and the transferability in performance of our model. However, the performance of the CNN framework falls short of expectations in various removal scenarios when the predicted task corresponds to a different network type than the one it was trained on, except for random node failures. Furthermore, our work suggests that directly predicting robustness values yields higher accuracy than capturing them through attack curve prediction. In addition, the observed scenario-sensitivity has been overlooked, and the transferability of predictive capability has been overestimated in the evaluation of network features in previous studies, necessitating further optimization. Finally, we discuss several important unresolved questions.