In constrained multi-objective optimization, the degree of constraint violation as an additional objective function has been optimized together with the original M objective functions for better diversity. However, it still faces the challenge of deeply exploring feasible regions while maintaining the diversity of the population. To this end, this paper proposes a novel constrained multi-objective evolutionary algorithm assisted by an additional objective function, called CMAOO. First, the main population is constructed to optimize an (M+1)-objective optimization problem consisting of the original M objective functions and the degree of constraint violation. Additionally, all the feasible solutions are saved in an external archive. Then, the main population and the external archive are evolved to search the whole space and the feasible regions, respectively. After that, their offspring are combined to separately update the external archive and the main population. Experimental studies are conducted to test the performance of CMAOO with four state-of-the-art algorithms on 34 test problems and a real-world problem. The results demonstrate that CMAOO is competitive to solve constrained multi-objective optimization problems.