Weather Translation via Weather-Cue Transferring

In this article, the weather translation task is proposed, which aims to transfer the weather type of the image from one category to another. Weather translation is a complicated image weather editing task that changes the weather cue of an image across multiple weather types, and it is related to image restoration, image editing, and photographic style transfer tasks. Although lots of approaches have been developed for traditional image translation and restoration tasks, only few of them are capable of handling the multicategory weather types problem with a single network due to the rich categories and highly complicated semantic structures of weather images. Especially, it is difficult to change the weather cue while preserving the weather-invariant area. To solve these issues, we developed a weather-cue guided multidomain translation approach based on StarGAN v2, termed WeatherGAN. In the proposed model, the core generator is redesigned to transfer the weather cue according to the target weather type. The weather segmentation module is first introduced to acquire the weather semantic structure of images in a weakly supervised multitask manner. In addition, a weather clues module is presented to reprocess the weather segmentation into a weather-specific clues map, which identifies the weather-invariant and weather-cue areas clearly. Extensive studies and evaluations show that our approach outperforms the state of the art. The data and source code will be publicly available soon after the manuscript is accepted.