When vision meets reality: Exploring the clinical applicability of GPT-4 with vision

In November 2023, OpenAI introduced the latest iteration of ChatGPT, which integrated a novel architecture called Generative Pre-trained Transformer (GPT) 4 with vision capabilities (GPT-4V). Different from the previous text-only architectures, GPT-4V is a "multimodal" large language model (LLM) capable of understanding both texts and images.1,2 In addition to the text corpora used in the training of previous GPT models, GPT-4V's training also included a vast collection of image and text caption pairings sourced from the internet.