Enhancing Programming Knowledge Tracing by Interacting Programming Skills and Student Code

Programming education has received extensive attention in recent years due to the increasing demand for programming ability in almost all industries. Educational institutions have widely employed online judges for programming training, which can help teachers automatically assess programming assignments by executing students' code with test cases. However, a more important teaching process with online judges should be to evaluate how students master each of the programming skills such as strings or pointers, so that teachers may give personalized feedback and help them proceed to the success more efficiently. Previous studies have adopted deep models of knowledge tracing to evaluate a student's mastery level of skills during the interaction with programming exercises. However, existing models generally follow the conventional assumption of knowledge tracing that each programming exercise requires only one skill, whereas in practice a programming exercise usually inspects the comprehensive use of multiple skills. Moreover, the feature of student code is often simply concatenated with other input features without the consideration of its relationship with the inspected programming skills. To bridge the gap, we propose a simple attention-based approach to learn from student code the features reflecting the multiple programming skills inspected by each programming exercise. In particular, we first use a program embedding method to obtain the representations of student code. Then we use the skill embeddings of each programming exercise to query the embeddings of student code and form an aggregated hidden state representing how the inspected skills are used in the student code. We combine the learned hidden state with DKT (Deep Knowledge Tracing), an LSTM (Long Short-Term Memory)-based knowledge tracing model, and show the improvements over baseline model. We point out some possible directions to improve the current work.