Designing an end-to-end encryption method for images using the nonlinear properties of deep neural networks (DNNs) has gradually attracted the attention of researchers. In this paper, we introduce a new framework for DNN-based image encryption that embeds a plaintext image as a secret message into a random noise to obtain a ciphertext image. Based on this, we propose an end-to-end robust image encryption method based on the invertible neural network (INN), which can realize secure encryption and resistance to common image processing attacks. Specifically, the INN is exploited as the shared-parameter encoder and decoder to achieve end-to-end encryption and decryption. The ciphertext image can be obtained through the forward process of the INN by inputting the plaintext image and the key, while the decrypted image can be obtained through the backward process of the INN by inputting the ciphertext image and the key. To enhance the security of our method, we design an information reinforcement module to guarantee the encryption effect and the sensitivity of the key. In addition, to improve the robustness of our method, an attack layer is employed for noise simulation training. Experimental results show that our method not only can realize secure encryption but also can achieve the robustness such as resisting JPEG compression, Gaussian noise, scaling, mean filtering, and Gaussian blurring effectively.