Recent progress in neural network estimation of atmospheric profiles using microwave and hyperspectral infrared sounding data in the presence of clouds

Recent work has demonstrated the feasibility of neural network estimation techniques for atmospheric profiling in partially cloudy atmospheres using combined microwave (MW) and hyperspectral infrared (IR) sounding data. In this paper, the global retrieval performance of the stochastic cloud-clearing / neural network (SCC/NN) method is examined using atmospheric fields provided by the European Center for Medium-range Weather Forecasting (ECMWF) and in situ measurements from the NOAA radiosonde database. Furthermore, the retrieval performance of the neural network method is compared with the AIRS Level 2 algorithm (Version 4). Comparisons of both forecast and radiosonde data indicate that the neural network retrieval performance is similar to or exceeds that of the AIRS Level 2 (version 4) profile products, substantially so in very cloudy areas. A novel statistical method for the global retrieval of atmospheric temperature and water vapor profiles in cloudy conditions has been developed and evaluated with sounding data from the Atmospheric InfraRed Sounder (AIRS) and the Advanced Microwave Sounding Unit (AMSU). The present work focuses on the cloud impact on the AIRS radiances and explores the use of Stochastic Cloud Clearing (SCC) together with neural network estimation. A stand-alone statistical algorithm will be presented that operates directly on cloud-impacted AIRS/AMSU data, with no need for a physical cloud clearing process. The algorithm is implemented in three stages. First, the infrared radiance perturbations due to clouds are estimated and corrected by combined processing of the infrared and microwave data using the SCC method. The cloud clearing of the infrared radiances was performed using principal components analysis of infrared brightness temperature contrasts in adjacent fields of view and microwave-derived estimates of the infrared clear-column radiances to estimate and correct the radiance contamination introduced by clouds. Second, a Projected Principal Components (PPC) transform is used to reduce the dimensionality of and optimally extract geophysical profile information from the cloud-cleared infrared radiance data. Third, an artificial feedforward neural network (NN) is used to estimate the desired geophysical parameters from the projected principal components. The performance of this method was evaluated using global (ascending and descending) EOS-Aqua orbits co-located with ECMWF fields for a variety of days throughout 2002 and 2003. Over 500,000 fields of regard (3x3 arrays of footprints) over ocean and land were used in the study. The NOAA radiosonde database was also used to assess performance - approximately 2000 global, quality-controlled radiosondes were selected for the comparison. The SCC/NN method requires significantly less computation (up to a factor of three orders of magnitude) than traditional variational retrieval methods, while achieving comparable global performance. Accuracies in areas of severe clouds (cloud fractions exceeding about 60 percent) is particular encouraging.