Concept mapping assessment in medical education: a comparison of two scoring systems

Background Concept mapping has the potential to measure important aspects of a student's evolving knowledge framework in a way that conventional examinations cannot. This is important because development of an elaborate and well-structured knowledge framework is a critical step toward becoming an expert in a particular field. Little is known about the best way to score concept maps in the setting of medical education. Therefore, as a preliminary step in addressing this question, we compared two different scoring systems for validity: a structural method based on the organization of a map's hierarchical structure and a relational method based, not on structure, but on the quality of each individual map component. Methods A total of 21 paediatric resident doctors completed concept map training, drew a preinstruction concept map about 'seizures', completed a seizure education course, and then drew a postinstruction seizure map. Two raters using both structural and relational methods scored each map. Results Structural scores increased significantly after instruction and were higher in more experienced residents, but relational scores were not significantly different. Interrater scoring reliability for both methods ranged from moderate to strong, but was greater using the relational scoring method. Conclusions These data suggest that scoring systems for evaluating concept maps in postgraduate medical education may need to account for structural features of maps, if scores are to reflect changes in the developing knowledge frameworks of resident doctors. More research to further evaluate reliability and validity is critical prior to any future use of concept mapping assessment in medical education.