Estimating confidence intervals for spatial hierarchical mixed-effects models with post-stratification

Analyzing population representative datasets for local level estimation and prediction purposes is important for monitoring public health, however, there are many statistical challenges associated with such analyses. Small area estimation (SAE) with post-stratified hierarchical mixed-effects models is a popular method for analysis. Post-stratification is a method that creates area-level predictions from a model fitting using sub-area-level covariates by incorporating auxiliary information (i.e., census data). While the post-stratification is an intuitive approach, the predictive benefits of post-stratification over standard methods with hierarchical mixed-effects models remain unclear. Another challenge for analyzing this type of data is the incorporation of sampling weights, as common data sources utilize complex sampling designs with uneven sampling probabilities. In addition, estimating the mean squared prediction error (MSPE) can be difficult via asymptotic theory due to the complex sampling designs and post-stratification process. Bootstrap methods can be an alternative, however there are many bootstrapping methods to choose from and their properties in realistic scenarios are unclear. In this paper, we compared the predictive ability of post-stratified and non-post-stratified estimators and evaluate the performance of various bootstrapping methods in estimating the MSPE with simulated data. Further, we compare the results using a population-based survey used to estimate the county-level prevalence of smoking in the state of South Carolina.