PROCESS OF CREATING A QUESTIONNAIRE TO ANALYZE TEACHER’S FEEDBACK: EXPERTS PANEL RESULTS

I. Armadans, A. Castrechini, C. Porrúa-García, C. Almendro

University of Barcelona, Department of Social Psychology and Quantitative Psychology (SPAIN)
Introduction:
One of the tasks made by university teachers is to provide feedback to students about their performance in the different assignments. Feedback is a process of fundamental importance in the teaching activity which has been numerous research efforts to ensure their optimization. This becomes the central task when tutoring students during master dissertation period. Nicol and MacFarlane (2006) presented a framework of seven principles of techniques to support good practice of feedback.

These principles are:
(1) clarify what good performance is;
(2) facilitate self-assessment;
(3) deliver high quality feedback information;
(4) encourage teacher and peer dialogue;
(5) encourage positive motivation and self-esteem;
(6) provide opportunities to close the gap; and
(7) use feedback to improve teaching.

Their work is based on the assumption that feedback should strengthen students’ capacity of self-regulate their own performance and contribute to their ability to learn for the long term. Consequently, the main aim of the present study was to create the items of an instrument for the self-diagnosis of the feedback provided by the teacher during the dissertation module of a master program. As a secondary aim, it was intended to analyze and validate the wording of the items.

Method:
The participants of this study were 8 tutors (5 women and 2 men) from a master's degree related to social and legal sciences. This panel of experts has an average experience of 3 years as tutor. Based on the seven principles of Nicole and MacFarlane model (2006), a list of 165 items was redacted, because each of the seven theoretical principles was divided into different subcategories; each of them refers to a type of strategy or aspect related to feedback.

Then these items were passed on to each of the tutors so that they could evaluate them according to three criteria:
(a) Relevance: if the item assesses what is to evaluate (scale: 1-9, where 1 is irrelevant and 9 is very relevant);
(b) Suitability of the item: it measure to what extent the item fits the theoretical dimension where is located (scale: 1-9, where 1 not suitable and 9 is very suitable);
(c) Clarity of the item (scale: 1-9, where 1 not understandable and 9 is very understandable).

The questionnaire also included a last section to collect additional comments or observations from the judges. RESULTS: The results show that some of the items have been scored with low values (scale: 1-3) in relation to Relevance (15.8 %), Suitability (13.3 %), and Clarity (10 %). The rest of the items has obtained scores considered as medium values (scale: 4-6) and high values (scale: 7-9). In relation to the items that have obtained low values, there are 10 items, 6 %, that have received low scores on three criteria. As a complement, it is emphasized that 54.5% of the items have been valued with a value lower than

Conclusions:
Instruments are fundamental tools, not only in research framework, but also in educational context. An instrument necessary for professional practice would be one that would serve to evaluate the quality of the feedback offered by the tutor. In this line is the research that is presented. The results obtained have been used to detect the items that must be eliminated to produce the first version of the questionnaire.

Acknowledgement:
This study is supported by the Institute of Educational Sciences (ICE) of University of Barcelona (Ref: REDICE16-1280).