THE USE OF SENSITIVITY ANALYSIS TO EVALUATE THE QUALITY OF A TEACHING ACTIVITY INDEX

Over the past few years, universities have undergone various changes, which have been largely conditioned by the demands of society. Universities have experienced a deep shift from an elite – oriented public, towards a university for the masses. Other factors that area gaining in importance in Higher Education systems are the increasing demands to improve quality, the globalization of knowledge, the increase in the competitiveness at national and international levels, the development of internal improvement procedures, and general changes in relationships with government and society are some of the factors that are gaining in importance in the field of Higher Education in Europe and around the world. Specifically, the purposes of the European convergence process include the establishment of criteria and methodologies to ensure the quality of university teaching.
The use of indicators as measurement tools can greatly assist with efficient management in a system as complex as any university is nowadays. In this case, indicators can be defined as objective, usually quantitative, measures for an institution or a whole higher education system. However, an indicator by itself rarely provides enough information to measure complex phenomena such as the teaching activity of a university’s teaching staff. These are multidimensional concepts whose measurement requires a set of indicators that reflect the various dimensions which define the concept to be measured. As a result, measuring multidimensional concepts requires a measurement tool known as a composite indicator (CI), which is a mathematical aggregation of a selected set of suitably weighted indicators.
Composite indicators have been increasingly adopted by many institutions for providing a measurement tool for attracting public attention and shaping integrated policy debates. However, on many occasions methodological neglect during their design, a lack of information and the use of inconsistent data may result in oversimplified conclusions, so the use of CIs are the subject of controversy. Composite indicator building should be accompanied by sensitivity analysis to ensure good practice. However, this is rarely done. Sensitivity analysis helps to improve the understanding and, ultimately, the soundness of the composite. In most cases, sensitivity analysis shows that the weights assigned to indicators do not reflect the actual importance of those indicators in the aggregation to the composite because of the heteroskedasticity of, and correlation between the underlying indicators. This paper proposes a composite indicator for the teaching activity of academic staff in a Spanish university. As we shall see in the paper, the desired weights stated by developers rarely represent the effective importance of the components. Hence, we propose sensitivity analysis as a necessary tool for re-adjusting weights in order to achieve the desired level of importance for each component indicator.