Learning Spatiotemporal Brain Dynamics in Adolescents via Multimodal MEG and fMRI Data Fusion Using Joint Tensor/Matrix Decomposition

Objective: Brain function is understood to be regulated by complex spatiotemporal dynamics, and can be characterized by a combination of observed brain response patterns in time and space. Magnetoencephalography (MEG), with its high temporal resolution, and functional magnetic resonance imaging (fMRI), with its high spatial resolution, are complementary imaging techniques with great potential to reveal information about spatiotemporal brain dynamics. Hence, the complementary nature of these imaging techniques holds much promise to study brain function in time and space, especially when the two data types are allowed to fully interact. Methods: We employed coupled tensor/matrix factorization (CMTF) to extract joint latent components in the form of unique spatiotemporal brain patterns that can be used to study brain development and function on a millisecond scale. Results: Using the CMTF model, we extracted distinct brain patterns that revealed fine-grained spatiotemporal brain dynamics and typical sensory processing pathways informative of high-level cognitive functions in healthy adolescents. The components extracted from multimodal tensor fusion possessed better discriminative ability between high- and low-performance subjects than single-modality data-driven models. Conclusion: Multimodal tensor fusion successfully identified spatiotemporal brain dynamics of brain function and produced unique components with high discriminatory power. Significance: The CMTF model is a promising tool for high-order, multimodal data fusion that exploits the functional resolution of MEG and fMRI, and provides a comprehensive picture of the developing brain in time and space.