Suicide Tendency Prediction from Psychiatric Notes Using Transformer Models

Suicide tendency is a fluid and multifaceted process that involves various stages, including suicidal ideation, planning, and attempting suicide. The use of electronic health records (EHR) and predictive algorithms has provided unprecedented opportunities for suicide research, but standard diagnosis codes for suicide tendencies are not always readily available in health records, resulting in low sensitivity when identifying suicide tendencies using structured data. Prior studies have focused on developing binary classification models to identify the presence of single suicide tendencies, such as suicide ideation or suicide attempt. In this study, we have worked on multiclass suicide tendency problem. We conducted a series of experiments to predict multiple suicide tendencies from psychiatric evaluation notes using classic machine learning models and pretrained transformer models. We manually annotated 1,000 Initial Psychiatric Evaluation (IPE) notes using a set of three classes (suicide ideation, suicide attempt, and non-suicidal). The performance of these models were evaluated using weighted F1 score, precision, recall, and accuracy. The Bio-ClinicalBERT model achieved the best performance for multiclass classification, with a weighted F1 score of 0.78, outperforming the classic machine learning models. Logistic regression and random forest models achieved comparable performance to state-of-the-art models in binary classification tasks with F1 score and accuracy of 0.93. The study contributes to mental health informatics with a novel Natural Language Paper (NLP) approach and psychiatric dataset.