U-DARTS: Uniform-space differentiable architecture search

Differentiable architecture search (DARTS) is an effective neural architecture search algorithm based on gradient descent. However, there are two limitations in DARTS. First, a small proxy search space is exploited due to memory and computational resource constraints. Second, too many simple operations are preferred, which leads to the network deterioration. In this paper, we propose a uniform-space differentiable architecture search, named U-DARTS, to address the above problems. In one hand, the search space is redesigned to enable the search and evaluation of the architectures in the same space, and the new search space couples with a sampling and parameter sharing strategy to reduce resource overheads. This means that various cell structures are explored directly rather than cells with same structure are stacked to compose the network. In another hand, a regularization method, which takes the depth and the complexity of the operations into account, is proposed to prevent network deterioration. Our experiments show that U-DARTS is able to find excellent architectures. Specifically, we achieve an error rate of 2.59% with 3.3M parameters on CIFAR-10. The code is released in https://github.com/Sun-Shiqi/U-DARTS.