Abstract:

The National Student Survey (UK) 2014 – 2016 for the School of Health Sciences (SHS) University of Nottingham consistently highlights a student experience of inequitable application of marking criteria and inadequate feedback. Given the implicit links between marking criteria and provision of feedback; improving and streamlining marking criteria practices was anticipated to impact positively on consistency and feedback. Student engagement in teaching and learning developments has become widespread; however not extensively with regard to summative assessment practice and procedures. Student engagement is generally suggested to increase student achievement of course learning outcomes, develop employer desired transferable skills and support mindfulness in learning opportunities. Whilst there are many potential issues with the philosophical role of summative assessments within Higher Education, this project looked to optimise current practice towards consistency rather than to revolutionise whole programs assessment approach. Strategies which look to reduce “fussiness” with simplicity, precision and communication of clear standards may support such consistency.

Recent work has been conducted in the SHS to streamline our marking criteria rubric to produce a weighted marking criteria domain (WMCD) rubric for marking of all written assessment. This work included a staff and student discussion workshop and an all staff and student questionnaire delivered via Bristol Online Survey. This staff- student collaboration led to the development of the WDMC rubric into which students and staff populated existing marking criteria statements.

The current project implemented this WMCD rubric and collated evaluation of staff and students response in terms of utility, feedback and technology use. The technology platform (Turnitin) was tested for its ability to produce a “predictive” mark based upon the WMCD domain selected by individual markers.

The aim of this project was to pilot the WMDC rubric with staff and students. Specific objectives were to;
- Use existing marking criteria to explore the rubric’s utility from staff and student perspectives
- Implement the WMCD with the electronic platform (Turnitin)
- Further the predictive mark capability of Turnitin would be explored

The school has four undergraduate profession entry program for Nursing, Midwifery, Physiotherapy and Sport Rehabilitation with a variety of Masters level postgraduate programs. The pilot study was carried out across three Divisions (Nursing, Midwifery and Physiotherapy) during the 2016 – 2017 academic year with 360 students assessments marked using the WMCD rubric. Markers were able to over-ride this predictive mark. Both the predictive and assessor mark were collected for analysis. There were no statistically significant differences between the predictive mark and assessor mark; Bland Altman analysis indicated strong alignment between marking methods as a school (bias=0.2). However, individual division Bland-Altman identified nuanced marker behaviour with bias of - 6.9, 0.75 & -0.9 (Midwifery, Nursing & Physiotherapy respectively). This abstract will consider both the predictive capability of the marker WMCD rubric to predict a mark and potential factors influencing divisional bias differences.

Keywords:

Marking, Assessment, Electronically Suggested Mark, Weighted Rubric.