Fusing multimodal data of nature-economy-society for large-scale urban building height estimation

The building height holds significant importance for comprehensively understanding urban morphology, enhancing urban planning, and fostering sustainable development. Although many methods using optical and SAR images have been presented for building height estimation, these methods fall short in capturing the influences of economic and social attributes on building height. In this study, we introduced a Nature-Economy-Society (NES) feature model to comprehensively represent building height information, and established a multi-scale One-Dimensional (1-D) Convolutional Neural Network for predicting building heights, referred to as NES-CNN. First, we derived the natural attributes of urban buildings from time-series Sentinel-1 SAR images and Sentinel-2 multispectral images, as well as World Settlement Footprint (WSF) data and Digital Elevation Model (DEM), economic attributes from nighttime light and Gross Domestic Product (GDP) data, and social function attributes from Points of Interest (POI) data. Second, an autoencoder is employed to reduce the dimensionality of the high-dimensional natural attribute features, minimizing data redundancy. Finally, the multi-scale 1-D CNN model is presented to explore the correlations between the multi-source and heterogeneous NES features and building height information, facilitating the prediction of building height. In experiments, we applied the proposed method to estimate building heights in Beijing and Shanghai at a spatial resolution of 10 m. The results indicated that for Beijing, the RMSE, MAE, and R values are 6.93 m, 4.41 m, and 0.84, respectively, while for Shanghai, these values are 7.57 m, 5.38 m, and 0.80, respectively. The addition of social and economic attribute information decreases the RMSE by 6 % in both Beijing and Shanghai compared with using only natural attributes. In comparison to existing studies at the same mapping resolution, RMSE decreases by 39 % for Beijing and 51 % for Shanghai. The innovative and inspiring nature of this study lies in its application to large-scale building height estimation.