Unmanned aerial vehicles (UAVs) combined with artificial intelligence (AI) have opened a revolutionized way for mobile crowdsensing (MCS). Conventional AI models, built on aggregation of UAVs' sensing data (typically contain private and sensitive user information), may arise severe privacy and data misuse concerns. Federated learning, as a promising distributed AI paradigm, has opened up possibilities for UAVs to collaboratively train a shared global model without revealing their local sensing data. However, there still exist potential security and privacy threats for UAV-assisted crowdsensing with federated learning due to vulnerability of central curator, unreliable contribution recording, and low-quality shared local models. In this paper, we propose SFAC, a secure federated learning framework for UAV-assisted MCS. Specifically, we first introduce a blockchain-based collaborative learning architecture for UAVs to securely exchange local model updates and verify contributions without the central curator. Then, by applying local differential privacy, we design a privacy-preserving algorithm to protect UAVs' privacy of updated local models with desirable learning accuracy. Furthermore, a two-tier reinforcement learning-based incentive mechanism is exploited to promote UAVs' high-quality model sharing when explicit knowledge of network parameters are not available in practice. Extensive simulations are conducted, and the results demonstrate that the proposed SFAC can effectively improve utilities for UAVs, promote high-quality model sharing, and ensure privacy protection in federated learning, compared with existing schemes.