Abstract:

Eye-tracking methodology offers a unique opportunity for assessing moment-to-moment comprehension processes, tracking the cognitive processes underlying visuospatial planning and problem solving. Eye-movement protocols have many important advantages over conventional methods. In a standardized test, questions can be asked in a context or symbolic representation. However how different presentations of the questions effect on students’ cognitive processes still remains as a question. Because of the eye movements are an integral part of the problem-solving process and solution steps are consequences of inner thought and attention, there is no need any additional effort on the part of the subject. Therefore, eye-tracking method gives us an opportunity to compare the cognitive effort of students who are trying to encode the questions in different modalities. The purpose of this study is to investigate the eye movement and pencil usage differences between symbolic representation of a question and context design in a passage. For this purpose, an experimental study was applied and forty-one students’ eye movements were recorded by an eye-tracker, their answers to sixteen GRE (Graduate Record Examination) questions were collected via an online assessment tool. Questions were asked through the browser and participants’ digital pen usage (mouse click counts) was recorded by graphic tablet via the software. Twenty-one participants took the Test-1 (two symbolic, six text passage) and the remaining twenty took the Test-2 (six text passage, two symbolic) which consisted of the same questions asked in a different presentation modality. Which participants would take the tests first was determined randomly. Tests were used in a counter-balanced design and the sequence of the questions were randomly aligned in order to eliminate the primary effect. Paired sample t-tests and Wilcoxon signed rank tests were applied on eye-metrics for the comparison of the presentation modalities. Results showed that eye movements and pencil usage were increased in text passages while eye movement metrics were decreased in symbolic modality. However, there was no significant difference between the questions with different contexts in terms of eye-movements and pencil usage. In addition, graphical visualizations showed that changing the presentation modality of a question causes changes in participants’ gaze behavior. Results will give an idea to the instructors and evaluators on designing questions with different modalities.

Keywords:

Question modality, problem solving, eye-tracking, pencil usage.