Improving urban tree species classification by deep-learning based fusion of digital aerial images and LiDAR

Accurate information on tree species distribution in urban areas can offer insights into how street trees provide ecosystem services, such as air pollution mitigation and surface cooling. This article presents a method to improve tree species classification in a tropical urban area using LiDAR-derived structural properties of individual tree crowns (ITCs) and digital aerial images. We extracted four LiDAR features, including surface normals of tree leaves, intensity, tree height, and leaf area index (LAI). We conducted two experiments: In the first, we trained encoder-decoder convolutional neural networks using a stack of optical bands and one LiDAR feature at a time. In the second, we developed an optical-LiDAR fusion strategy that combined feature maps from two encoder-decoder networks. One network was trained with optical bands only, while the other was trained with the LiDAR features that improved classification accuracy in the first experiment. Our experiment results demonstrated the usefulness of surface normals and intensity in discriminating among tree species. We found that the optical-LiDAR fusion strategy increased the average F1-score by 12.6 percentage points compared to only optical bands. We also employed the new segment anything (SAM) model to automatically delineate ITCs. SAM outlined ITCs with a boundary F1-score of 98%. The SAM-delineated ITCs were used to improve raw model predictions and produce reliable species maps. This study contributes to mapping and monitoring urban tree species in tropical areas.