This paper discusses the consensus problem for a kind of nonlinear fractional-order multi-agent system (FOMAS) by employing the passivity of every agent. Distinguishing from the integer-order multi-agent systems, a concept of passivity for FOMAS is given. On one side, a passivity criterion is presented for the FOMAS by resorting to the Lyapunov functional method. On the other side, with the help of the Laplace transform, a consensus criterion for the FOMAS is obtained under the condition that the FOMAS is output-strictly passive. Lastly, the effectiveness of the obtained criterion is verified by a numerical example with simulations.