Abstract:

Most Adaptive Exercise Systems use a domain model to specify the structure of their learning domain. Quite often the learning topics in this domain model are not sufficiently detailed. Consequently, the exercise system cannot personalize the exercises in such a way it can guarantee that all different facets of each learning topic are trained.

Therefore, we designed a low level methodology for the automated refinement of a learning topic into several learning subtopics. More specifically, we start from a set of exercises on the same learning topic and group all exercises having a common learning subtopic. The resulting classification can be used to detail the domain model, by adding the learning subtopics found below the original learning topic in the domain model. In the remainder of this abstract, we resume our methodology and describe how we validated our approach.

Our methodology starts from a set of exercises all training the same learning topic. It requires all exercises to have a pre-calibrated difficulty and to be answered by the same group of learners (logging data). In a first step, we estimate the ability of each learner for the learning topic, by a maximum likelihood estimation based on the Item Response Theory (IRT). In a second step, we use the results from the first step, IRT and statistical hypothesis testing to calculate a similarity measure for each pair of exercises. The calculated measure correlates to the chance that both exercises train the same learning subtopic. In calculating this measure, we exploit the fact that a learner does not master all learning subtopics of the learning topic equally well, meaning there is some intervariability in his abilities for the different learning subtopics. The resulting measures define a graph of exercises with similarity measures. We cluster this graph by an agglomerative hierarchical clustering algorithm. This finally results in a grouping defining what exercises train the same learning subtopic.

We have validated our methodology by programming and interpreting a complete simulation. We defined a set of exercises, each having a random difficulty and learning subtopic, and a set of learners with random abilities. We simulated the answering of the exercises by the learners, based on the Rasch prediction model of IRT. Next, we calculated the similarity measures and clustered the exercises into groups. We compared the resulting groups with the expected results. Simulation results point out we require approximately 500 learners to split up 50 exercises into four groups reliably. The number of learner’s required increases with the number of exercises, the number of groups to detect and the inverse of the intervariability of the learner’s abilities for the learning subtopics. We also tested the robustness of our methodology against several kinds of noise such as perturbations in the Rasch model.

Keywords:

Domain model, adaptive exercise system, learning topic.