On Comparing Early and Late Fusion Methods

This paper presents a theoretical comparison of early and late fusion methods. An initial discussion on the conditions to apply early or late (soft or hard) fusion is introduced. The analysis show that, if large training sets are available, early fusion will be the best option. If training sets are limited we must do late fusion, either soft or hard. In this latter case, the complications inherent in optimally estimating the fusion function could be avoided in exchange for lower performance. The paper also includes a comparative review of the fusion state of the art methods with the following divisions: early sensor-level fusion; early feature-level fusion; late score-level fusion (late soft fusion); and late decision-level fusion (late hard fusion). The main strengths and weaknesses of the methods are discussed.