Enhancing stock movement prediction with market index and curriculum learning

Stock movement prediction is a complex and challenging task, because the real stock market system is sophisticated and noisy. There are two obstacles with directly using historical trading data to predict stock price movement: (1) the stock market is time-varying, resulting in the mismatch between training and test data and poor generalization performance; (2) the stock market is a noisy system in which price moving signal is difficult to capture. To overcome these two challenges, we propose a method enhancing stock movement prediction with market index and curriculum learning. We consider the impact of the market movement on individual stock movement, and propose to use market index to split the impacts of market movement and intrinsic individual stock movement. In response to the noise issue, we propose two hypotheses: the trading mode deviation hypothesis and the price prediction uncertainty hypothesis. Based on these two hypotheses, we separately use autoencoder and Mixture Density Network (MDN) to obtain Trading Mode Deviation (TMD) and Price Prediction Uncertainty (PPU) of samples. TMD and PPU are correlated with the difficulty of samples and can be used to assess the uncertainty of price movement. Then, we propose to use curriculum learning algorithm to train model. The experimental results of the CSI Smallcap 500 index (CSI500) stock trading data show that the Micro-F1 of our method has been improved and our method effectively alleviates the class imbalance.