Coh-Metrix: Capturing Linguistic Features of Cohesion

Abstract This study addresses the need in discourse psychology for computational techniques that analyze text on multiple levels of cohesion and text difficulty. Discourse psychologists often investigate phenomena related to discourse processing using lengthy texts containing multiple paragraphs, as opposed to single word and sentence stimuli. Characterizing such texts in terms of cohesion and coherence is challenging. Some computational tools are available, but they are either fragmented over different databases or they assess single, specific features of text. Coh-Metrix is a computational linguistic tool that measures text cohesion and text difficulty on a range of word, sentence, paragraph, and discourse dimensions. This study investigated the validity of Coh-Metrix as a measure of cohesion in text using stimuli from published discourse psychology studies as a benchmark. Results showed that Coh-Metrix indexes of cohesion (individually and combined) significantly distinguished the high- versus low-cohesion versions of these texts. The results also showed that commonly used readability indexes (e.g., Flesch–Kincaid) inappropriately distinguished between low- and high-cohesion texts. These results provide a validation of Coh-Metrix, thereby paving the way for its use by researchers in cognitive science, discourse processes, and education, as well as for textbook writers, professionals in instructional design, and instructors. Notes aEfficiency refers to performance and reaction time. bSkill refers to reading skill. cCould not be computed because data provided in the article were all correlational. dInference test refers to open-ended inference questions answered while referring to the text. eQuestions are open-ended.