REAL WORLD USAGE OF AN ADAPTIVE TESTING ALGORITHM TO UNCOVER LATENT KNOWLEDGE
There are now many intelligent learning environments that can guide students through the learning materials of a course or curriculum. However before a student begins a new course, it is reasonable to expect they might have some prior knowledge of the learning materials. This knowledge could have been obtained from many sources; typical examples include taking related courses elsewhere or accessing the wealth of educational materials online.
To promote learning efficiency, it is crucial that each student’s latent knowledge can quickly be uncovered by the learning environment. This enables students to begin learning new material immediately and not have to repeat materials already known to them. By personalizing the course in this way, the learning system avoids much potential frustration and boredom on the part of the student.
To implement this, we developed an algorithm called ‘Determine Knowledge’ that exploits the structure of the curriculum to efficiently uncover the latent knowledge of a student. This functionality is offered as an adaptive pretest at the beginning of each course. The students who selected this option are then automatically placed at the appropriate position within the course as determined by the algorithm.
In this paper, we detail the algorithm and review its relative strengths and weaknesses. In particular we discuss the real-world usage of Determine Knowledge across 575 course instances run in the United States over a period of two years. In total this encompasses over 455,000 Determine Knowledge operations performed with 48,500 unique students. From this data, we investigate how this algorithm was utilized in reality and we evaluate its performance across subject domains.