Abstract:

The National Distance Education University (UNED), at more than 205,000 students, has the largest student population in Spain and is one of the largest universities in Europe. Since the teaching is at distance, one of the objectives of the University is to analyze the results obtained by the students and to reinforce those aspects of the subjects that are more difficult. In this way, this work is included in a project in which the results obtained by the students in the tests of the degrees in Computer Science are analyzed to detect those topics that are more difficult.

The aim of this work is to present a methodology based on Artificial Intelligence techniques to automatically identify the elements that are most complicated for students to learn in the different subjects involved in the project. By means of this methodology we will be able to extract and statistically analyze the different topics that are part of a course and thus optimize the improvement obtained with the production of new materials, focusing on those elements that are harder for the students to learn.

This methodology automatically identifies the main topics covered during the course and allows for a statistical analysis of the results of evaluation tests from past courses by relating the topics of the questions to the results. In this sense, a generative model on the content of the subjects (Latent Dirichlet allocation) has been designed and implemented to automatically obtain the different existing topics.

The results shown in this work prove the great efficiency of this methodology. To carry out the evaluation, a dataset of test questions manually labelled by the teachers was used, assigning to each question the topic of the syllabus to which it corresponds. The system presented in this work detects mostly the topic of the syllabus to which each question corresponds, obtaining a great correlation with the topics assigned by the teachers. These results make it possible to generalise the automatic detection of topics in university subjects. Moreover, each of these topics is represented by a set of keywords, which allows teachers not only to detect the different topics but also to clearly identify the parts of the curriculum to which they refer.

These techniques will be evaluated on some of the subjects taught by the authors for which we have information collected over the years. However, preliminary results show that this methodology can be exported to any other university subject.

Keywords:

Education technologies, natural language processing, challenging topics, subjective perception, learning analytics.