ANALYSIS OF RELIABILITY AND VALIDITY OF THE PERFORMANCE ASSESSMENT SYSTEM FOR STUDENTS OF THE SECOND-LEVEL STUDIES IN NURSING - A SEVEN-YEAR RETROSPECTIVE ANALYSIS

Introduction:
Educational measurement at the higher education level is now based on the assessment of a student’s progress, with particular emphasis on the measurement of learning outcomes. Evidence-based assessment, as a good practice, should be applied in any well manager higher education institution which attempts to adjust its educational policy to the changing needs and requirements of the labour market.

Aim of study:
Analysis of the reliability and validity of the system of assessing students’ performance in subjects terminating with an examination at the second-level studies in the field of nursing.

Materials and Methods:
Examination data of 940 students in the field of nursing who undertook second-level studies at the Faculty of Health Sciences, Medical University of Warsaw (MUW) in the years 2006-2012 were qualified for the study. Retrospective analysis covered educational performance in ten subjects which terminated with an examination: Management in nursing, Principles of psychotherapy, Theory of nursing, European nursing, Epidemiology, Medical didactics, Psychiatry, Medical services, Family planning, Law in health care.
The internal cohesion of the assessment system was evaluated with alpha-Cronbach coefficient. The concordance of the educational measurement for individual subjects was evaluated with the use of ANOVA Friedman non-parametric test with Kendall’s coefficient of concordance. The theoretical validity assessment was performed with the use of two methods: 1) r-Spearman’s rank correlation coefficient to estimate the inter-correlation of educational performance; 2) exploratory factor analysis with Varimax rotation of raw factor loadings.

Results:
The total measurement reliability for the ten analysed subjects was alpha-Cronbach = 0.605. The total comparative analysis of mean grades obtained by students in the years 2006-2012 reveals that the worst assessed area was Epidemiology (mean grade 3.2 ± 0.73), while the best Principles of psychotherapy (mean grade 4.8 ± 0.40). For all the remaining subjects students’ mean grades ranged from 3.6 to 4.4 which indicates an average level of concordance (Tau Kendall’s coefficient 0.34). Results of the internal structure of students’ performance assessment show that two subjects Principles of psychotherapy and Medical services do not show significant positive inter-correlations with the remaining subjects. The assessment of validity by means of the factor analysis revealed that four subjects form separate domains of the assessed educational performance. The strongest distinctness was registered in the case of Principles of psychotherapy while a medium level of distinctness was observed in Medical services, Psychiatry and Theory of nursing.

Conclusions:
The estimated reliability and validity of the applied methods of nursing students’ performance assessment allows to validate and further monitor the quality of the assessment system. In future, it will be necessary to evaluate the criterion validity by estimating the prognostic capacity of the principles of assessment with the use of data concerning further professional careers of graduates.