Time series is mostly with a chaotic nature and non-stationary characteristic in real-word, which makes it difficult to be modeled and predicted accurately. To solve this problem, we introduce a novel self-organizing modular neural network based on the empirical mode decomposition with the sliding window mechanism (SWEMD-MNN) for time series prediction. In SWEMD-MNN, the improved empirical mode decomposition with sliding window (SWEMD) is developed to decompose time series online, which can effectively alleviate the limitation that the traditional EMD-based models cannot handle the long term or online problem and end effect. Thus, SWEMD-MNN can decompose time series based on time characteristic effectively and dynamically, and improve the prediction accuracy of the classical modular neural networks dividing time series based on sample space. Then time subseries are dynamically assigned to the subnetworks with a single layer feedforward neural network using the sample entropy and Euclidean distance for learning. Experimental investigations using benchmark chaotic and real-world time series show that SWEMD-MNN can decompose time series effectively and dynamically, and provides a better prediction accuracy than the fully coupled networks and other MNN models for time series prediction.