UTILIZING A MULTICRITERIA ADAPTIVE MULTIMEDIA TOOL IN FORMATIVE ASSESSMENTS

Adaptive testing systems are computerized tests aiming to accurately reflect the instructional level of each student with a shorter number of queries tailored to the characteristics of the individual learner. The existing adaptive systems have proved to be beneficial as the assessments tend to be shorter and more accurate. However they base their adaptation mainly on the learner’s performance using various statistical models. Basing the adaptation strategy to other learner characteristics, such as their aims and their preferences, would help educators gain a better understanding of their students and students to form a clearer idea of their knowledge and difficulties.

In this paper we present and utilize an adaptive testing tool, called iAdaptTest (interoperable Adaptive Testing tool). Personalization is based on a set of rules which concern the knowledge, performance, goals and preferences of learners. The system maintains user profiles for learners and educators. In these profiles several attributes describing the users are maintained. These attributes vary from personal details, to user knowledge and to user preferences. The testing items are questions in textual or in a multimedia form. Each test is associated with a specific topic of a domain (e.g. Electromagnetism in Physics). Adaptation is achieved through the customization of pre-existing rules. The run time environment takes as input all the aforementioned and executes the test, adapting the sequence of the presented items according to the tester’s replies and the adaptation rules. Finally, testers are presented with their score and analytic descriptions and conclusions about their progress.

The described system offers a rich set of personalization options and thus extensive evaluations are needed to realize its strengths and potential limitations. A first round of evaluations was run with the aid of 10 teachers and of 25 high school and 30 college students.

The focus of the evaluation of the teachers was to understand how the teachers feel about such a system and whether they can produce various types of adaptive assessments to support their educational aims. The educators were asked to prepare some formative for their classes. Following the construction of the assessments, a short focused interview was conducted separately with each participant in order to record their beliefs about the system.

Then the system was used in assessing the knowledge of college students and high school students. The main aim of these assessments was to help students understand their current knowledge on different algorithmic topics they have been taught and to help the teachers see the misconceptions of their students. Different rules, based on the previous knowledge, the performance and the preferences of the students were utilized. At the end of the assessments the dynamically formed testing paths of the students were studied in order to understand which rules were initiated and to measure the pedagogical gains of the students with respect to forming a clearer idea of their knowledge.

The paper is structured as follows. Section 1 is the introductory section. Different adaptivity methods in e-assessment tools are critically reviewed in section 2. Next, in Section 3, the design and the features of iAdaptTest are presented. The results of the evaluations are presented in Section 4 and Section 5 presents the main conclusions of the paper and proposes future research directions.