In dealing with expensive constrained multi-objective optimization problems using surrogate-assisted evolutionary algorithms, it is a great challenge to reduce the negative impact caused by the approximate errors of surrogate models for constraints. To address this issue, we propose a Kriging-assisted evolutionary algorithm with two search modes to adaptively reduce the utilization frequency of surrogate models for constraints. To be more specific, an adaptively switching strategy analyzing the correlation between the objective optimization direction and constraint satisfaction direction is designed to determine whether to build the constraint surrogate models to assist the current evolutionary search. Accordingly, the proposed algorithm contains two search modes: 1) unconstrained surrogate-assisted search mode and 2) constrained surrogate-assisted search mode. In the first search mode, an existing surrogate-assisted evolutionary algorithm without considering constraint is introduced, which rapidly drives the population to move to the feasible region(s) while avoiding the negative effects of the constraint surrogate models. In the second search mode, a novel Kriging-assisted constrained multi-objective optimization algorithm is designed for locating constrained Pareto front in the feasible region. In addition, a data selection strategy is proposed to improve the efficiency and quality of surrogate models for constraint functions. The proposed method has been tested on numerous instances from three popular benchmark test suites. The experimental results demonstrate that the performance of the proposed algorithm outperforms other state-of-the-art methods.