Abstract:

Modern Bologna-based curricula in the European Higher Education Area (EHEA) is based on the acquisition of technical and professional competencies. Technical competencies are mainly acquired at the University altogether with initial professional competencies. Lifelong learning is expected to further increase the alumni portfolio of professional competencies during their career. Considering the curricula in the University and further on in the professional career, it is interesting to introduce a fully competency-based evaluation for technical and professional qualitative assessment.
Coherent alumni ranking in heterogeneous studies is of great importance nowadays in Spain due to the current degree structure. Bachelor (named “Grado” in Spain) and Master levels are nation-wide stablished in all Universities. Alumni with Bachelor degrees can enrol in any Master from a wide range of public and private Universities. Government-stablished public Universities must fulfil the requirement of equal-right to access to Master-level studies strictly based on student qualifications. It is then necessary to stablish a ranking system enabling adequate comparison between the different Bachelors in order to effectively enable universal equal access rights to Master degree.
This paper proposes and reports a first trial implementation of a new qualitative assessment technique based on identifying the missing competencies in examination tests. With this technique, quantitative evaluation is derived from carefully evaluated weights on the assessed and missing competencies. Adequate University-level competency-weighting permits to rank alumni achievements in a coherent way considering heterogeneous students from different Engineering Technical Schools in the Universitat Politècnica de València (UPV).
A multi-dimensional evaluation is performed for the qualitative assessment of carefully identified technical and academic competencies and an adequate weighting is stablished. Assessment of the student’s competencies gives insight into the actual performance as the educational objectives are carefully designed at all levels in the curricula to achieve concrete academic and technical competencies including basic knowledge and professional skills.
The proposed technique is demonstrated in the academic year 2013-14 considering both quantitative and qualitative ranking in the subject Communications Theory (“Teoría de la Comunicación”) taught in second year of the Integrated Program (Bachelor and Master) “Ingeniero de Telecomunicación” in the UPV. The assessment employs different evaluation methodologies: short theoretical questions, multiple-choice tests, guided-solving problems and complex developing problems. The qualitative assessment points out different kinds of faults that the student can face during the knowledge building process, i.e. fundamental errors, comprehension mistakes, analytical errors, theoretical misunderstanding or application difficulties.
The evaluation trial comprises a group of students during this academic year and ranks alumni in a Gaussian distribution centred in 5 over 10 point score. The competency-weights derived from the statistical ranking give relevant information on the most and less successful competencies developed in the group, which is of great importance for the teachers. Moreover, good correlation has been observed between the Gaussian distribution marks and the final scores obtained by the students employing the conventional quantitative ranking.

Keywords:

Competency assessment, qualitative evaluation, alumni ranking, best practices