MGCNet: Multilevel Gated Collaborative Network for RGB-D Semantic Segmentation of Indoor Scene

RGB-D semantic segmentation of indoor scenes has long been an enduring research topic. However, because of the intrinsic differences in modal information and large gaps in multi-level feature cues, adopting the traditional U-Net framework provides suboptimal indoor scene segmentation. In this paper, we consider an effective feature exploration approach to achieve accurate segmentation. Specifically, it consists of three steps. First, in the encoder, we design a difference-exploration fusion module, which extracts the difference weights of the two modalities to guide them for fusion, so as to achieve intrinsically consistent feature fusion. The gated decoder module relates to the remaining two steps. Second, we use a gating unit for each level of fusion information to reduce the difference between layers, which also increases the unique distinction of a specific layer while avoiding the exclusion between layers of information. Finally, we use a serial-parallel alternation strategy to increase the ability to capture contextual knowledge. Considering the above three steps, we construct the multilevel gated collaborative network (MGCNet). Extensive experiments indicate the performance of the proposed MGCNet can compete favorably against state-of-the-art models under three standard metrics.