The Weibull distribution is the most widely applied model in reliability analysis. The main objective of this paper is to present a simple method called the minimum discrepancy method that is applicable to both complete and censored data for the parameter estimation of the Weibull distribution and a detailed comparison in a small sample size of thirteen methods in terms of several criteria by a simulation study. Additionally, parameter estimation methods are applied to the spring fatigue failure data. By extensive simulations and comparisons, the generalized least square 1, the weighted least square 1, the weighted Maximum likelihood estimation and the minimum discrepancy method are recommended for parameter estimation with small samples.