Maximum Likelihood from Incomplete Data Via the EM Algorithm

Journal of the Royal Statistical Society: Series B (Methodological)Volume 39, Issue 1 p. 1-22 ArticleFree Access Maximum Likelihood from Incomplete Data Via the EM Algorithm A. P. Dempster, A. P. Dempster Harvard University and Educational Testing ServiceSearch for more papers by this authorN. M. Laird, N. M. Laird Harvard University and Educational Testing ServiceSearch for more papers by this authorD. B. Rubin, D. B. Rubin Harvard University and Educational Testing ServiceSearch for more papers by this author A. P. Dempster, A. P. Dempster Harvard University and Educational Testing ServiceSearch for more papers by this authorN. M. Laird, N. M. Laird Harvard University and Educational Testing ServiceSearch for more papers by this authorD. B. Rubin, D. B. Rubin Harvard University and Educational Testing ServiceSearch for more papers by this author First published: 1977 https://doi.org/10.1111/j.2517-6161.1977.tb01600.xCitations: 8,788AboutPDF ToolsExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Summary A broadly applicable algorithm for computing maximum likelihood estimates from incomplete data is presented at various levels of generality. Theory showing the monotone behaviour of the likelihood and convergence of the algorithm is derived. Many examples are sketched, including missing value situations, applications to grouped, censored or truncated data, finite mixture models, variance component estimation, hyperparameter estimation, iteratively reweighted least squares and factor analysis. References Andrews, D. F., Bickel, P. J., Hampel, F., Huber, P. J., Rogers, W. H. and Tukey, J. W. (1972). Robust Estimates of Location. Princeton, N.J.: Princeton University Press. Baum, L. E. (1971). An inequality and associated maximization technique in statistical estimation for probabilistic functions of Markov processes. In Inequalities, III: Proceedings of a Symposium. ( Qved Shisha ed.). New York: Academic Press. Baum, L. E. and Eagon, J. A. (1967). An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology. Bull. Amer. Math. Soc., 73, 360– 363. Baum, L. E., Petrie, T., Soules, G. and Weiss, N. (1970). A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Statists. 41, 164– 171. Beale, E. M. L. and Little, R. J. A. (1975). Missing values in multivariate analysis. J. R. Statist. Soc., B, 37, 129– 145. Blight, B. J. N. (1970). Estimation from a censored sample for the exponential family. Biometrika, 57, 389– 395. Brown, M. L. (1974). Identification of the sources of significance in two-way tables. Appl. Statist., 23, 405– 413. Carter, W. H., Jr and Myers, R. H. (1973). Maximum likelihood estimation from linear combinations of discrete probability functions. J. Amer. Statist. Assoc, 68, 203– 206. Ceppellini, R., Siniscalco, M. and Smith, C. A. B. (1955). The estimation of gene frequencies in a random-mating population. Ann. Hum. Genet., 20, 97– 115. Chen, T. and Fienberg, S. (1976). The analysis of contingency tables with incompletely classified data. Biometrics, 32, 133– 144. Corbeil, R. R. and Searle, S. R. (1976). Restricted maximum likelihood (REML) estimation of variance components in the mixed model. Technometrics, 18, 31– 38. Day, N. E. (1969). Estimating the components of a mixture of normal distributions. Biometrika, 56, 463– 474. Dempster, A. P. (1972). Covariance selection. Biometrics, 28, 157– 175. Efron, B. (1967). The two-sample problem with censored data. Proc. 5th Berkeley Symposium on Math. Statist. and Prob., 4, 831– 853. Efron, B. and Morris, C. (1975). Data analysis using Stein's estimator and its generalizations. J. Amer. Statist. Assoc, 70, 311– 319. Good, I. J. (1965) The Estimation of Probabilities: An Essay on Modern Bayesian Methods. Cambridge, Mass.: M.I.T. Press. Good, I. J. (1956). On the estimation of small frequencies in contingency tables. J. R. Statist. Soc., B, 18, 113– 124. Grundy, P. M. (1952). The fitting of grouped truncated and grouped censored normal distributions. Biometrika, 39, 252– 259. Haberman, S. J. (1976). Iterative scaling procedures for log-linear models for frequency tables derived by indirect observation. Proc. Amer. Statist. Assoc. (Statist. Comp. Sect. 1975), pp. 45– 50. Hartley, H. O. (1958). Maximum likelihood estimation from incomplete data. Biometrics, 14, 174– 194. Hartley, H. O. and Hocking, R. R. (1971). The analysis of incomplete data. Biometrics, 27, 783– 808. Hartley, H. O. and Rao, J. N. K. (1967). Maximum likelihood estimation for the mixed analysis of variance model. Biometrika, 54, 93– 108. Harville, D. A. (1977). Maximum likelihood approaches to variance component estimation and to related problems. J. Amer. Statist. Assoc, 72, to appear. Hasselblad, V. (1966). Estimation of parameters for a mixture of normal distributions. Technometrics, 8, 431– 444. Hasselblad, V. (1969). Estimation of finite mixtures of distributions from the exponential family. J. Amer. Statist. Assoc, 64, 1459– 1471. Healy, M. and Westmacott, M. (1956). Missing values in experiments analysed on automatic computers. Appl. Statist. 5, 203– 206. Hosmer, D. W. Jr (1973). On the MLE of the parameters of a mixture of two normal distributions when the sample size is small. Comm. Statist., 1, 217– 227. Hosmer, D. W. Jr (1973). A comparison of iterative maximum likelihood estimates of the parameters of a mixture of two normal distributions under three different types of sample. Biometrics, 29, 761– 770. Huber, P. J. (1964). Robust estimation of a location parameter. Ann. Math. Statist., 35, 73– 101. Irwin, J. O. (1959). On the estimation of the mean of a Poisson distribution with the zero class missing. Biometrics, 15, 324– 326. Irwin, J. O. (1963). The place of mathematics in medical and biological statistics. J. R. Statist. Soc., A, 126, 1– 45. Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183– 202. McKendrick, A. G. (1926). Applications of mathematics to medical problems. Proc. Edin. Math. Soc., 44, 98– 130. Mantel, N. and Greenhouse, S. W. (1967). Note: Equivalence of maximum likelihood and the method of moments in probit analysis. Biometrics, 23, 154– 157. Maritz, J. S. (1970). Empirical Bayes Methods. London: Methuen. Martin, J. K. and McDonald, R. P. (1975). Bayesian estimation in unrestricted factor analysis: A treatment for Heywood cases. Psychometrika, 40, 505– 517. Mosteller, F. and Wallace, D. L. (1964). Inference and Disputed Authorship: The Federalist. Reading, Mass.: Addison-Wesley. Orchard, T. and Woodbury, M. A. (1972). A missing information principle: theory and applications. Proc. 6th Berkeley Symposium on Math. Statist. and Prob. 1, 697– 715. Patterson, H. D. and Thompson, R. (1971). Recovery of inter-block information when block sizes are unequal. Biometrika, 58, 545– 554. Raiffa, H. and Schlaifer, R. (1961). Applied Statistical Decision Theory. Cambridge, Mass.: Harvard Business School. Rao, C. R. (1965). Linear Statistical Inference and its Applications. New York: Wiley. Rubin, D. B. (1974). Characterizing the estimation of parameters in incomplete-data problems. J. Amer. Statist. Assoc, 69, 467– 474. Rubin, D. B. (1976). Inference and missing data. Biometrika, 63, 581– 592. Sundberg, R. (1974). Maximum likelihood theory for incomplete data from an exponential family. Scand. J. Statist., 1, 49– 58. Sundberg, R. (1976). An iterative method for solution of the likelihood equations for incomplete data from exponential families. Comm. Statist–Simula. Computa., B5(1), 55– 64. Turnbull, B. W. (1974). Nonparametric estimation of a survivorship function with doubly censored data. J. Amer. Statist. Assoc, 69, 169– 173. Turnbull, B. W. (1976). The empirical distribution function with arbitrarily grouped, censored and truncated data. J. R. Statist. Soc., B, 38, 290– 295. Wolfe, J. H. (1970). Pattern clustering by multivariate mixture analysis. Multivariate Behavioral Research, 5, 329– 350. Woodbury, M. A. (1971). Discussion of paper by Hartley and Hocking. Biometrics, 27, 808– 817. Citing Literature Volume39, Issue11977Pages 1-22 This article also appears in:Discussion Papers ReferencesRelatedInformation