Image all-in-one adverse weather removal via dynamic model weights generation

Restoring image under multiple weather conditions in an all-in-one fashion remains a formidable challenge due to images captured under different weather conditions exhibit different degradation characteristics and patterns. However, existing all-in-one adverse weather removal methods mainly focus on learning shared generic knowledge of multiple weather conditions via fixed network parameters, which fails to adjust for different instances to fit exclusive features characterization of specific weather conditions. To tackle this issue, we propose a novel dynamic weights generation network (DwGN) that can adaptively mine and extract instance-exclusive degradation features for different weather conditions via dynamically generated convolutional weights. Specifically, we first propose two fundamental dynamic weights convolutions, which can automatically generate optimal convolutional weights for distinct pending features via a lightweight yet efficient mapping layer. The predicted convolutional weights are then incorporated into the convolution operation to extract instance-exclusive features for different weather conditions. Building upon the dynamic weights convolutions, we further devise a tailored weight adaptive Transformer blocks (WATB) which consists of two core modules: half-dynamic multi-head cross-attention (HDMC) that performs exclusive-generic feature interaction, and half-dynamic feed-forward network (HDFN) that performs selected exclusive-generic feature transformation and aggregation. Considering communal features shared between different weather conditions (e.g., background representation), both HDMC and HDFN deploy only half of the dynamic weights convolutions for instance-exclusive feature characterization, while still deploying half of the static convolutions to characterize generic features. Through adaptive weight tuning, our DwGN can adaptively adapt to different weather scenarios and effectively capture the instance-exclusive degradation features, thus enjoying better flexibility and adaptability under all-in-one adverse weather removal. Extensive experiments demonstrate that our DwGN performs favorably against state-of-the-art algorithms. In particular, our proposed DwGN achieves the best PSNR and SSIM scores on all five tasks both in the task-specific setting and in the all-in-one setting. Furthermore, our method has shown consistent performance improvement in both real-world and high-level visual applications. The implementation code is available at https://github.com/Jeasco/DwGN.