3D Md-Unet: A novel model of multi-dataset collaboration for medical image segmentation

• The multi-dataset collaborative network can process different organs or lesions for medical image segmentation at the same time. • The proposed adapter (SSA) can extract specific and common features from multiple classes within a dataset and various datasets. • The proposed adaptive weight update strategy can balance multi-dataset better, which is based on classes instead of voxels. • The dual-branched (DB) structure is more effective than the single one for multi-dataset collaboration. Image segmentation is widely used in the medical field. Convolutional neural network has become more diverse and effective in recent years. However, at present, most networks are designed for a single dataset (i.e., a single organ or target). The designed network is only suitable for a single dataset, and its accuracy is very different (especially small-size image datasets). In response to this problem, a collaborative network can be designed to simultaneously extract the specific and common features of a multi-dataset (i.e., multiple organs or targets). The network can be used for multi-dataset segmentation and help to balance the segmentation performance of different datasets, especially to improve the accuracy of small-size image datasets. By exploring the adapters modified by the convolution kernels, the adaptive weight update strategy and the network branched structure, the paper proposes a multi-dataset collaborative image segmentation network, called Md-Unet, which integrates a shared-specific adapter (SSA), an asymmetric similarity loss function with the proposed adaptive weight update strategy, and a dual-branch. Experimental results showed that compared with the baseline 3D U 2 Net, the accuracy of the module using the SSA was improved by 3.7%, using several loss functions with the proposed adaptive weight update strategy was improved by 0.64%–30.63%, and using dual-branch integrated architecture was improved by 17.47%. Moreover, Md-Unet had a significant improvement on small-size image datasets compared with single-dataset models.