The impact of student clustering on the results of statistical tests

This chapter discusses the effect of clustering on statistical tests and illustrates this effect using science laboratory classroom environment data. Because much classroom research involves the collection of data from students who are nested within classrooms, the hierarchical nature to these data cannot be ignored. In particular, this chapter studies the influence of intraclass correlations on tests of statistical significance conducted with the individual as the unit of analysis. Theory that adjusts t test scores for nested data in two-group comparisons is presented and applied to science laboratory environment data. This chapter demonstrates that Type I error rates inflate greatly as the intraclass correlation increases. Data analysis techniques that recognise the clustering of students in classrooms are essential and it is recommended that either multilevel analysis or adjustments to statistical parameters be undertaken in studies involving nested data.