Abstract:

This paper shares partial findings of a three-year research project, funded by the National Science Foundation, that examined undergraduate students’ metacognitive knowledge about task and self-regulation deployed during problem-solving activity. Qualitative and quantitative analyses were used to develop understandings of students’ self-evaluation skills of their problem-solving performance in two undergraduate engineering and mathematics (EM) courses. Self-evaluation skills are essential components of one’s self-regulatory capacity in problem solving. The major research question constructed to guide this study was: How accurately do undergraduate students judge the quality of their worked solutions while solving EM problems?

Two 2nd year EM courses, Engineering Statics and Ordinary Differential Equations, were purposefully selected to be the contexts of the study. Qualitative data were generated with 20 undergraduate students (i.e., 7 females, and 13 males) across both courses (i.e., 11 and 9 students from mathematics and engineering, respectively) through one-on-one interviews. During data generation, each student engaged in four EM content-driven problem-solving activities of varying levels of difficulty (i.e., 2 relatively easy and 2 relatively difficult activities) and at two different instances in time (i.e., early and later in the semester. Data generation resulted in a total of 80 problem-solving qualitative data generation events with 20 unique participants.

Students’ problem-solving solutions were evaluated by the instructors teaching the two courses. The qualitative data from the interviews were analyzed to capture students’ perceptions about their confidence level solving the problems prior to problem-solving activity and their self-evaluative judgments about the solutions after solving the problems. Quantitative data analyses were then conducted to evaluate students’ self-evaluative judgments through calculating some essential measures of problem-solving perceived performance such as students' sensitivity and specificity scores, absolute correctness index (ACI), and judgment accuracy index (JAI).

During data analysis, we found:
(1) participants evaluated their problem-solving performance more accurately after, rather than before they engaged in the activity;
(2) students were excessively confident on self-judging for correctness, as reflected in their ACI, when solving easier, compared to difficult problems;
(3) more accurate self-evaluation were made by students solving mathematics problems than those students solving engineering problems; and
(4) students’ self-evaluative judgments improved as the semester progressed. Furthermore, students reported that when judging their problem-solving performance, they appraised not only the final answers, but also the procedure, to get to those answers. It was also reported that some of the students acknowledged the mistakes in the solutions. Brief discussion is included at the end of the paper.

Keywords:

Engineering education, mathematics education, problem solving, self-evaluation, self-judgments, self-regulation.