Transformer-based deep learning models for adsorption capacity prediction of heavy metal ions toward biochar-based adsorbents

Biochar adsorbents synthesized from food and agricultural wastes are commonly applied to eliminate heavy metal (HM) ions from wastewater. However, biochar's diverse characteristics and varied experimental conditions make the accurate estimation of their adsorption capacity (qe) challenging. Herein, various machine-learning (ML) and three deep learning (DL) models were built using 1518 data points to predict the qe of HM on various biochars. The recursive feature elimination technique with 28 inputs suggested that 14 inputs were significant for model building. FT-transformer with the highest test R2 (0.98) and lowest root mean square error (RMSE) (0.296) and mean absolute error (MAE) (0.145) outperformed various ML and DL models. The SHAP feature importance analysis of the FT-transformer model predicted that the adsorption conditions (72.12%) were more important than the pyrolysis conditions (25.73%), elemental composition (1.39%), and biochar's physical properties (0.73%). The two-feature SHAP analysis proposed the optimized process conditions including adsorbent loading of 0.25 g, initial concentration of 12 mg/L, and solution pH of 9 using phosphoric-acid pre-treated biochar synthesized from banana-peel with a higher O/C ratio. The t-SNE technique was applied to transform the 14-input matrix of the FT-Transformer into two-dimensional data. Finally, we outlined the study's environmental implications.