Abstract:

Conceptual maps were first proposed by Novak [1] a few decades ago. They consist of a very flexible structure that allows creating meaningful relationships between key concepts so as to improve the processes of learning and knowledge acquisition in general. Since their invention, they have received much attention from researchers and experts worldwide. In fact, conceptual maps have been used in various areas of knowledge with heterogeneous purposes such as creating cross-language tools, thereby simplifying concept translation between different languages, as an aid in solving interpersonal conflicts, as a tool to enhance web searches [6], or even to improve the public outreach of the Mars exploration mission.
Evaluating students using conceptual maps is quite interesting since it allows them to externalize their own mental trees of assimilated concepts to a simple format, thereby offering the instructor an Ausubelian perspective of the knowledge acquired. Adopting such a student-centred evaluation strategy also helps at detecting misunderstandings and flaws in the students' learning process.
One of the first works in the field of evaluation using conceptual maps was proposed by Cronin et al. [2] in 1982. More recently, Temple and Marshall have proposed an evaluation strategy based on the number of concepts and the number of links appearing in the conceptual map. Authors such as Ozdemir [3] have undergone a validation of conceptual maps by comparing them against more traditional evaluation techniques; in his work, the author showed that the scores obtained by students using either method were rather similar.
We consider that one of the main drawbacks when evaluating conceptual maps has to do with the subjectiveness in associating a score with one particular conceptual map. In fact, different instructors evaluating the same set of conceptual maps are prone to assign different scores within a relatively wide range of values. Additionally, if the evaluation process is repeated by the same instructor, results are also expected to differ to a certain extent, which evidences the variability of traditional methods.
In this paper we seek to introduce more objectiveness into the evaluation process by proposing a strategy that aids instructors at evaluating conceptual maps. We apply the proposed strategy in a Computer Networks course for undergraduate students of a Degree on Documentation. The technique basically consists in splitting the evaluation process into different steps, and defining a set of criteria and heuristics to evaluate each of these steps. The proposed fragmentation of the evaluation process takes into consideration the different parts involved in creating conceptual maps themselves. We then derive a formula that aggregates the individual scores in a comprehensive manner.
Taking a set of conceptual maps created by our students as reference, we show that the systematic approach of our proposal helps at reducing the variability in scores, thereby improving the fairness of the evaluation process and also reducing the amount of effort undertaken by the instructor.

References:
[1] Novak, J. D., A theory of education, Cornell University Press, Ithaca, NY, 1977.
[2] Patrick J. Cronin et , al, A procedure for using and evaluating concept maps, Springer Research in Science Education, 1982
[3] Ozdemir, A., Analyzing Concept Maps as an Assessment (Evaluation) Tool in Teaching Mathematics, Journal of Social Sciences, Vol. 1, No. 3, 2005,

Keywords:

concept maps, evaluation techniques, objective metrics.