Fusion of EEG and fMRI via Soft Coupled Tensor Decompositions

Data fusion refers to the joint analysis of multiple datasets which provide complementary views of the same task. In this paper, the problem of jointly analyzing electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI) data is considered. Analyzing both EEG and fMRI measurements is highly beneficial for studying brain function because these modalities have complementary spatiotemporal resolutions: EEG offers good temporal resolution while fMRI offers good spatial resolution. The fusion methods reported so far ignore the underlying multi-way nature of the data in at least one of the modalities and/or rely on very strong assumptions concerning the relation among the respective data sets. In this paper, these two points are addressed by adopting tensor models for both modalities and by following a soft coupling approach to implement the fused analysis. To cope with the subject variability in EEG, the PARAFAC2 model is adopted. The results obtained are compared against those of Parallel ICA and hard coupling alternatives in both simulated and real data. Our results confirm the superiority of tensorial methods over methods based on ICA. In scenarios that do not meet the assumptions underlying hard coupling, the advantage of soft coupled decompositions is clearly demonstrated.