Validating a Test to Measure Depth of Vocabulary Knowledge

This chapter reports on the final stage of an investigation into the English for specific purposes (ESP) claim that tertiary level students for whom English is a second language (ESL) should be given reading proficiency tests in their own academic subject areas. A previous report on the research described how reading passages in the International English Language Testing System (IELTS) test, which has modules in three academic subject areas, varied in their subject specificity: Some were suitable for students in the relevant academic field; some were either too general or too specific (Clapham, 1996a). This chapter specifically compares the relative contributions of background knowledge and language proficiency to the students' test scores, and investigates the extent to which the effect of background knowledge on reading comprehension varies according to students' level of language proficiency. First, the results of multiple regression analyses showed that language proficiency accounted for more of the test score variance than did background knowledge, but that the comparative effects of the variables differed according to the subject specificity of the tests. When the analysis was based on students' scores on the complete reading modules, language proficiency accounted for almost half of the variance, but when the students' scores were based on revised tests from which the nonspecific subtests had been removed, language proficiency had less effect, and background knowledge 142proportionately more. This suggests that the comparative importance of background knowledge and level of language ability in reading comprehension depends on the specificity of the reading passages. Second, analysis of variance was used to compare the effect of background knowledge on the reading performance of students at different levels of language proficiency. Results showed that the reading scores of low-proficiency students were not affected by background knowledge, but that the scores of students at higher levels of proficiency were. There was no steady increase in the effect as the students' ability levels rose. Instead, there appeared to be a level of proficiency below which students were unable to make use of their background knowledge. This supports Clarke's (1980) hypothesis that there is a threshold level of language proficiency that must be reached before students can make use of their top-down reading processes. The effect of test specificity on these results and the implications for English for academic purposes (EAP) testing are presented in this chapter.