Abstract:

Evaluation is a complex task that requires the definition of clear and transparent criteria, so that all stakeholders understand evaluation as a credible act, of responsibility and educational and social utility. However, evaluation is not an exact science, so subjectivity is an inherent feature of this entire process. In higher education, there are few studies on assessment and in mathematics they are almost non-existent. It is pertinent that a greater reflection on the knowledge of evaluation be initiated in higher education institutions.

The SOLO Taxonomy is a cognitive methodological tool developed by Biggs and Collis. This methodology is based on a classification system composed of 5 levels of learning complexity, which consider aspects of quality and quantity of information:
1) prestructural;
2) unistructural;
3) multistructural;
4) relational; and
5) extended abstract.

The contributions of the SOLO Taxonomy to educational evaluation are numerous. In the evaluation of exams, the taxonomy, in addition to contributing to the elaboration of items, provides more complete results (involving the level of cognitive complexity of students' thinking) to teachers. So, the SOLO Taxonomy can assist in the preparation of exam items, which allows teachers to balance superficial and deep issues, and can also be used to analyse test results (pedagogical interpretation).

Over the past few years, the authors have applied the SOLO taxonomy to evaluate tests and exams. In this paper, it will be showing the process followed to construct test or exam questions in accordance with the SOLO taxonomy. Each exercise was catalogued according to the methodology proposed by [1] in 5 stages: Classification criteria; Proposed resolution; Topics, Procedures and Categorization. In the first stage, the evaluation criteria are defined by grouping common references to be considered in the evaluation, clearly describing their characteristics, and assigning their score, so that the evaluation is consistent and impartial. The second stage allows for a rigorous description of the item's resolution with the necessary justifications. Subsequently, all program topics and subtopics are identified. A set of procedures must be followed to reach a solution to the item. Sub questions were defined that guide the student in completing the exercise and allow teachers to study the level of complexity of the SOLO taxonomy that the student can reach. The aim is to assess the student's knowledge through solving the exercise. To this end, conceptual (or mental) maps were developed to organize the topics in a coherent way and identify the complexity of mathematical thinking, as support and structure for developing the resolution of the item. Finally, by analysing how themes and concepts are recognized, differentiated, and grouped into an item, it is possible to establish their categorization at SOLO level.

Examples of the construction of exercises will be presented based on their categorization into complexity levels of the SOLO taxonomy and how the student's level of knowledge is measured on the SOLO level scale. The degree/cognitive difficulty of the proposed exercise will be measured and whether the student meets the skills required to solve it using the SOLO index, with the aim of better understanding how and in what way mathematics students should be assessed in higher education.

Keywords:

SOLO Taxonomy, assessment, mathematics, higher education.