Abstract:

In recent years, the number of trend analyses in education is considerably increased. In order to ensure the comparability of collected data over time, a pre-requisite of large-scale (both sample-based and population-based) educational assessment is measurement invariance of the testing instruments over time. This pre-requisite is one of the main properties of the Rasch model.

This research project pursues two main goals. The first one is carrying out a longitudinal study on student performance. The National and International trend analyses conducted until now are cross-sectional and not longitudinal studies because they use data collected by administering tests to subjects in equivalent groups but not following the same students over time.

Starting from the academic year 2011/2012, INVALSI gathers panel data, by administering an anchor test to a subset (composed by the same students) of the National sample. So, the first two interesting aspects of our proposal are:
1. The availability of an innovative dataset (the first one in Italy);
2. The opportunity of carrying out a longitudinal analysis in the correct manner.

Diachronic studies are preferable compared to cross-sectional ones because, when sample composition changes, we cannot taking into account the students’ history. So, it is not possible to control the influence exerted by some variables (i.e. socio-economic status of students’ family) on the evolution of students’ performance. Moreover, those variables can influence the probability of a correct answer and so they change the estimation of ability parameters and so the item ordering on the continuum. When this happens, we have to hypothesize that the testing instruments are not unidimensional and so that the Rasch model is not adequate to order items and subjects on the latent trait. In order to individuate the factors that influences students’ performance (and also the magnitude of their effect), the study of Differential Item Functioning (DIF) can be carried out. DIF regards to “differential” item behaviors as a function of different criteria (i.e. gender, place of residence, socio-economic status of students’ family, etc.) to classify subjects on the latent trait.

The study of differential item functioning can also be used to ensure that testing instruments hold over time (the second aim of our study). The administration of INVALSI anchor test is aiming at developing an instrument to maintain constant the difficulty level of test. Of course, in a longitudinal perspective, “constant” does not mean that the difficulty level of items administered to the same subjects in different waves has to be constant over time (on the contrary, the difficulty level has to increase over time to appreciate the evolution of students’ performance). Vice versa, in order to ensure that the difficulty level of testing instruments are constant in each wave for each class (involved in the analysis and composed by different students year by year) must be constant over time according to the difficulty required by each scholastic level.

Keywords:

Differential Item Functioning, Rasch Model, Testing Instrument Dimensionality, Longitudinal Analysis of Students' Performance.