Abstract:

After implementing the principles of the European Higher Education Area, the evaluation of all students’ learning must be supported by three basic elements: initial evaluation, participatory evaluation, and summative evaluation. The summative evaluation has a wide implantation and is part of the organization culture in most higher education institutions, as it is most traditional approach and most common and widely employed techniques are part of it. The initial evaluation is very common in some countries, such as the Anglo-Saxon area or the Northern Europe, and it basically studies the students’ expectations, their knowledge level, their motivation, etc. Nevertheless, this evaluation system is not as common in the Mediterranean area as in other regions. Any case, initial evaluation is an increasing practice, and no technical barriers prevent its implementation, a significant effort is not required from professors, and it is not necessary a consensus or coordination involving several people. However, as stated in the Bologna declaration, the main component of students’ learning evaluation in higher education should be the participatory evaluation and, sadly, it is most neglected or misunderstood component.

Actually, the participatory evaluation faces several open challenges. On the one hand, most subjects should be organized to follow a competence-based scheme and approach. The continuous monitoring and improvement process is not feasible if an exhaustive and specific list of required competences is not available. On the other hand, student must attend in-person sessions regularly to monitor their progress and daily work. But even in innovative approaches, the students’ motivation for attending and participating in activities reduces linearly as time passes. The combination of these two challenges is causing a huge damage to participatory evaluation, so the evaluation (at the end) collapses and reduces to a sequence of tests or presentation. But these tests follow the summative evaluation principles, not the participatory evaluation ones. Actually, some authors have reported a reduction in the students’ learning quality as the number of summative tests increases.

In order to address this situation and enable a good quality participatory evaluation, a pilot experience was planned and carried out at Universidad Politécnica de Madrid. As a proposal, during the second term of the academic years 2020/21 and 2021/22, we have organized an international cybersecurity competition. In this competition could also participate students through the EELISA project and community. The competition was based on a collection of challenges to be solved with an original and creative proposal by group of students. For every competence in the program, at least two challenges were posed. Solutions were presented in-person and validated by professors and other students. Once finished all the courses, students were asked to fill a survey about the experience and their perception about the process and their learning. Academic results of students, the participation rates in the competition and other similar statistics were processed and analyzed.

Results confirmed this approach enables a better participatory evaluation. Students’ participation and attendance rates grew up in a significant manner and academic results also improved. The level of competence acquisition was also higher, as well as the students’ motivation

Keywords:

Cybersecurity, educational competitions, challenge-based learning, gamification, participatory evaluation, international experiences.