The authors examined whether Deep-IRT's student-ability predictions exhibit two problems reported for DKT: decreases in predicted performance after a correct response and inconsistent predictions across knowledge components over time. Deep-IRT produced smooth prediction transitions, although it still generated several irrelevant predictions.

San Diego State University

Google

1. Going Deeper in Difficulty Level
2. Going Deeper in Student Ability

Discussion (Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory)

The authors evaluated several ways of measuring item difficulty. First, they used the FSAIF1toF3 dataset, whose publisher supplied difficulty levels for each question, to assess how well Deep-IRT's estimated difficulty matched these labels. Second, as an alternative, difficulty was measured from how many students answered each question correctly versus incorrectly, counting only each student's first attempt and only questions with at least 10 respondents. Third, they used PFA together with each student's full learning trajectory to estimate difficulty, restricting this analysis to a subset of five skills. The authors suggest it would be worthwhile to test whether Deep-IRT's difficulty estimates are more accurate than these traditional approaches.

Learn Before

Related