Comparing the performance of global, geographically weighted and ecologically weighted species distribution models for Scottish wildcats using GLM and Random Forest predictive modeling

Species distribution modeling has emerged as a foundational method to predict occurrence and suitability of species in relation to environmental variables to advance ecological understanding and guide conservation planning. Recent research, however, has shown that species-environmental relationships and habitat model predictions are often nonstationary in space, time and ecological context. This calls into question modeling approaches that assume a global, stationary ecological realized niche and use predictive modeling to describe it. This paper explores this issue by comparing the performance of predictive models for wildcat hybrid occurrence based on (1) global pooled data across individuals, (2) geographically weighted aggregation of individual models, (3) ecologically weighted aggregation of individual models, and (4) combinations of global, geographical and ecological weighting. Our study system included GPS telemetry data from 14 individual wildcat hybrids across Scotland. We developed predictive models both using Generalized Linear Models (GLM) and Random Forest machine learning to compare the performance of these differing algorithms and how they compare in stationary and nonstationary analyses. We validated the predicted models in four different ways. First, we used independent hold-out data from the 14 collared wildcat hybrids. Second, we used data from 8 additional GPS collared wildcat hybrids from a previous study that were not included in the training sample. Third, we used sightings data sent in by the public and researchers and validated by expert opinion. Fourth, we used data collected by camera trap surveys between 2012 – 2021 from various sources to produce a combined camera trap dataset showing where wildcats and wildcat hybrids had been detected. Our results show that validation using hold-out data from the individuals used to train the model provides highly biased assessment of true model performance in other locations, with Random Forest in particular appearing to perform exceptionally (and inaccurately) well when validated by data from the same individuals used to train the models. Very different results were obtained when the models were validated using independent data from the three other sources. Each of these three independent validation data sets gave a different result in terms of the best overall model. The average of independent validation across these three validation datasets suggested that the best overall model produced for potential wildcat occurrence and habitat suitability was obtained by an ensemble average of the global Generalized Linear Model (GLM) and Random Forest models with the ecologically weighted GLM and Random Forest models. This suggests that the debate over whether which of GLM vs machine learning approaches is superior or whether global vs aggregated nonstationary modeling is superior may be a false choice. The results presented here show that the best prediction applies a combination of all of these approaches in an ensemble modeling framework.