What counts? The critical role of qualitative data in teachers’ decision making

The push for data-based decision making in schools has largely centered on the use of quantitative data to inform technical-rational processes of teachers' decision making. Previous attention to teachers' reliance on qualitative data - particularly unsystematically collected qualitative data – tends to focus on their use of intuition and is often characterized as a counter to evidence based inquiry. Limited research, however, has been conducted to understand how teachers actually apply available data within their classrooms, the factors that shape teachers' decision making, or what they consider credible in assessing their students' progress and achievement. In this collective case study, 15 teachers from three high schools discuss how they exercise professional judgment and make instructional decisions based on qualitative evidence. It takes an intentionally grounded approach to exploring the many data points that teachers draw upon as they face decision moments in their daily practice. In interviews and observations over the course of one school year, teachers describe various types of qualitative data that shed light on students' experiences as they undertake processes of learning. As these teachers glean bits and pieces of systematically and unsystematically collected qualitative data, including informal, undocumented data through conversations and observation, these data inspire reflective questioning and hypotheses about their instructional practice. While student progress should not be wholly assessed based on qualitative data, the findings show that we must acknowledge the inevitable and critical role these data play in guiding teachers' actions and informing their professional judgment. The persistent integration of qualitative data - though sometimes pointed to as a threat to rational decision-making processes – instead confirms a reliance upon them in guiding classroom instruction and a need to ensure their appropriate use.