The increasing demand for privacy protection facilitates growing interests in Federated Learning (FL). Nevertheless, most of existing FL schemes are susceptible to malicious participating clients compromised by Byzantine attacks, which remains a challenging issue. In this paper, we propose a novel Byzantine-robust FL scheme, coined FLPhish. Specifically, we first design a ensemble learning-based FL architecture, named Ensemble Federated Learning (Ensemble FL). Second, a phishing mechanism is crafted for the FL architecture to detect abnormal client behaviors. Third, a reputation mechanism is developed to further identify malicious participating clients compromised by Byzantine attackers. We evaluate the performance of FLPhish by considering various fractions of Byzantine clients and various imbalance degrees of the data distribution. Extensive experiments demonstrate the high effectiveness of the proposed FLPhish scheme in resisting Byzantine attacks in Ensemble FL.