Assessing the ethical capabilities of Chat GPT in healthcare: A study on its proficiency in situational judgement test

This study examines the proficiency of Chat GPT, an AI language model, in answering questions on the Situational Judgement Test (SJT), a widely used assessment tool for evaluating the fundamental competencies of medical graduates in the UK. A total of 252 SJT questions from the Oxford Assess and Progress: Situational Judgement Test book were sampled, encompassing 82 multiple-choice and 170 ranking questions. Chat GPT exhibited a commendable mean accuracy of 77.67% with a standard error of 1.09% when compared to the book's answers. While precise population statistics and a definitive scoring system remain unavailable, it is worth acknowledging the AI's consistent performance across all five domains tested. To significantly enhance its effectiveness and practical utility, especially in aiding junior doctors with complex ethical dilemmas, further advancements are imperative to strengthen its decision-making capabilities beyond factual comprehension.