In recent years, with the development of medical image analysis, the image processing of brain tumor and stroke has been deeply studied.However, when processing clinical medical imaging data with different characteristics or information collected from different sensors or modalities, that is, multimodal imaging data, the segmentation accuracy is low.Therefore, the research is based on generative adversarial networks and three-dimensional residual U-shaped networks to study brain tumor and stroke image generation and lesion segmentation.Experimental results showed the three models performed best in various conversions.For example, in T1→ Flair conversion, the generative multi-modal image analysis model based on perceptual loop consistency had an average peak signal-to-noise ratio of 23.951 ± 2.735, an average structural similarity of 0.873 ± 0.046, and an average root mean square error of 16.998 ± 6.184.All three models significantly raised the segmentation effectiveness of lesions, such as the combination of dual-scale perceptual loop generation adversarial network and three-dimensional residual U-shaped network for generative multi-modal image generation and lesion segmentation algorithm.Using three real input modalities, its HD index value of 75.082 and precision index value of 0.696 were better than the HD index value of 84.776 and Precision index value of 0.686.In addition, the study also conducted ablation experiments on a generative multi-modal image analysis model based on dual-scale perceptual loop consistency, indicating that the cavity residual module is hoped to have a good influence on lesion segmentation.Overall, the algorithm model proposed in the study has high effectiveness in the generation and segmentation of brain tumors and stroke images, and is of great significance for the development of medicine.