Abstract:

Academic literacy is an essential skill in university development. Today, not only a technical profile is required, but a good development of communication skills. In the Chilean Engineering programs, it is observed that the results of the students in these soft skills are deficient. For this, a digital tool is proposed that allows obtaining a diagnosis of the reading level and its metacognitive strategies.

In this context, it is necessary to obtain a reliable measurement of reading skills in students, particularly in engineering. For this, a new psychometric instrument called "Diagnostic Test: Reading Comprehension Strategies for University Students" (PD-ELCU) is proposed. The objective is to prepare a test with adequate psychometric characteristics (in terms of validity, reliability, homogeneity and representation) that allows capturing the entry reading profile of first-year students at Chilean universities. Examining the reading profile in the comprehension aspects implies that the instrument is sensitive in capturing the strengths, weaknesses and potentials of the candidates or applicants to enter the higher education system.

Traditionally, the practice of evaluation in the university has been interpreted as a measurement. The exclusive use of this type of test as a practice out of context, meaningless for the student, for summative purposes, does not correspond to the required constructive, cognitive and metacognitive process. Consequently, there is a need to rethink the concept of evaluation at the university level, with the aim of putting it at the service of developing self-regulatory capacities in students.

The main elements of the analysis of results are psychometric in nature: validation through expert judgment, internal consistency, difficulty and discrimination of items and judgments of the participating subjects regarding the levels of difficulty of the texts, clarity of instructions, execution time for understanding and metacognitive strategic judgments.

The results allow us to establish a relationship between the behavior of the readers when they face the texts and the metacognitive judgments issued. Additionally, these results may guide group and individual intervention. For the internal consistency analysis, the Cronbach's alpha statistic was used. Its value was 0.76, which explains a high internal consistency. Regarding the other analyzes, we worked with the difficulty index and the discrimination index. The difficulty or ease index (If) of an item is the proportion of subjects that guess it out of the total of those who have tried to solve. The discrimination index (Id) distinguishes between subjects with high and low scores on the test, based on the correlation between the scores on the item and their scores on the test.

As a conclusion, the test we propose differs significantly from the classic one, since it allows examining the strategic potential of the students when they face reading tasks, which is critical in the field of skills developed in engineering. The item information material may be similar, but the objectives are completely different, highlighting the evaluation of metacognition and its processes (planning, supervision and evaluation). When readers put these processes into practice while reading, they empower their capacity for self-learning and self-regulation.

Keywords:

Comprehension, Reading, Cognition, Evaluation, Engineering.