Abstract:

Engineering laboratory sessions are designed to foster conceptual understanding and practical competence, yet instructors often lack timely, traceable evidence of what students learn within a single hands-on class. This study reports an educational innovation implemented in a third-year Materials Engineering laboratory course (Mechanical Engineering degree) based on short formative quizzes administered immediately before (pre-test, pre) and after (post-test, post) each session.

Each laboratory session used a 10-item multiple-choice quiz aligned with the intended learning outcomes. Quizzes were delivered through Socrative (a web-based classroom response system) with a single attempt and randomized question and answer order to reduce pattern effects and copying. Feedback and grades were released only after the post-test to preserve the validity of the pre–post comparison. Raw exports were processed to compute participation indicators, handle missing data, and quantify learning gains per session and across the course. The analysis combined paired pre–post comparisons with statistical models that account for clustered observations because the same students contributed data across multiple sessions, and it was complemented with item-level learning analytics (difficulty, discrimination, and pre–post changes).

Across 10 sessions, 389 matched student-session pairs were available for the main analysis. Mean performance increased from 8.697 to 9.347 correct answers out of 10, corresponding to a mean gain of +0.650 (95% bootstrap confidence interval [0.530, 0.769]) and a repeated-measures effect size of d_z = 0.542. Global models controlling for session effects and within-student dependence yielded a consistent post effect of about +0.65 points (for example, +0.655, 95% confidence interval [0.501, 0.809]; p < 0.001). After multiple-comparison control, significant gains were observed in 7 of the 10 sessions, with the largest improvement in an ultrasound session (P08-1, +1.385/10). The median normalized gain, excluding cases with no room for improvement (pre = 10), was 0.600 (n = 256). Item-level analysis identified questions with strong positive shifts and also detected one item with a marked post-test drop in a session (P07), providing actionable evidence to revise explanations, lab guides, and distractors. A ceiling effect was present (10/10 increased from 34.19% in pre to 56.30% in post), suggesting the need for more discriminating questions in future item banks.

Overall, the proposed workflow is low-overhead and reproducible, provides direct evidence of immediate learning during laboratory work, and supports continuous improvement by linking session design, learning evidence, and targeted instructional adjustments.

Keywords:

Formative assessment, pre-post test, engineering laboratory, learning analytics, Socrative, materials engineering education.