Enhancing hydrological data completeness: A performance evaluation of various machine learning techniques using probabilistic fusion imputer with neural networks for streamflow data reconstruction

The present-day accessibility of streamflow data, particularly in the developing countries, is often marked by a multitude of data shortfalls or distortions. This study investigates the estimate of missing streamflow data using machine learning approaches, including K-nearest neighbour (KNN), Predictive Mean Matching (PMM), Random Forest (RF) and a novel technique of Probabilistic Fusion Imputer with Neural Networks (PROFINN). This study tackles the issue of data insufficiency in such time series considering the inclusion of numerous hydrological parameters by means of RF feature selector, to determine the most significant ones among them. The study explores the efficacy of selected models on various data gaps under different hydrological scenarios, including diverse flow characteristics (mean, high and low flows) and gap lengths (long and short gaps, continuous and discontinuous gaps) and presents a pattern of ranking system that assesses the level of suitability of each technique for various data gaps. This study underscore PROFINN's remarkable performance across all scenarios, yielding an average Root Mean Square Error (RMSE) of 0.91 and an average Nash-Sutcliffe Efficiency (NSE) value of 0.93 when applied for an intermittent river system of Pamba in Southern Kerala, India. RF follows PROFINN in the imputation of extreme flows as well as long and short gap scenarios. KNN closely follows PROFINN for the imputation of continuous and discontinuous gap scenarios. This study augments the significance of tailored machine learning techniques in enhancing the integrity of hydrological datasets, offering valuable insights for effective decision-making in water resource management and related fields. Furthermore, the success of PROFINN in streamflow data imputation suggests its potential extension to other hydrological research domains, like historical data reconstruction assisting climate change studies and future water resources planning, emphasizing the broader relevance and applicability of this study's findings.