Machine Learning Under Partial Observability: The Effect Of The SEC's Labeling Bias On Detecting Financial Misconduct

Traditional statistical methods and newer machine learning methods are used to identify predictors of financial misconduct periods. However, the partial observability of committed financial misconduct biases these prior findings. That is, it is crucial to not only consider misconduct firms labeled by the labeling mechanism but also account for unlabeled financial misconduct. In this paper, we use machine learning methods incorporating modeling partial observability to improve the identification of predictors' ability to identify financial misconduct periods. We exploit predictors used in previous literature to model financial misconduct and gather an extensive data set consisting of various new features to capture the labeling propensity. We exploit ensembles that are tailored to handle the inherent small sample size of financial misconduct observations. We conduct various empirical analyses to investigate the machine learning models' decision behaviors, enabling us to strengthen or update our beliefs about past findings and illustrate the good performance of our methodology.