Abstract:

The global development and widening availability of information and communication technology (ICT) has made digital skills essential for social interaction, civic participation, information retrieval and processing, academic performance, and professional success. Consequently, these skills should be considered as educational outcomes that students nowadays need to achieve, and therefore need to become in central role in curriculums in every educational level. This stresses the need for valid assessment instruments in order to measure these skills in education.

The new national core curriculum for basic education in Finland aims to offer opportunities for pupils to develop their information and communication technology skills within all subjects. This paper concentrates on measurement of ICT skills and presents the background and characteristics of the instrument called the ICT skill test, which is a tool for assessing ICT skills. This paper offers a brief overview of the ICT skills among upper comprehensive school students in Finland. Actual research goals in this paper were to confirm, by means of item analysis, the adequacy of the test and to examine what kind of effect the different item characteristics have on the item difficulty.

The paper discovers that students performed relatively well in items related to social networking and communication, content creation and using productivity software. Instead, the items which required more technical skills, like programming, basic operations or information networks, were found to be challenging for the majority of the students. Girls were found to be more proficient than boys in communication and social networking, content creation and using productivity software. Boys in their turn dominated the items of basic operations, information networks, software purchasing, installing and updating applications, and all items related to programming. Still, even the majority of the boys were found to perform inadequately in items requiring operational and technical knowledge.

By means of the item analysis, the ICT skill test was found to be reliable as a whole and its items being appropriately formed and having acceptable discrimination power. The examination of item characteristics exposed unexpectedly that question text length, neither the stem or option length, had no impact on item difficulty. The presence of images was found to slightly increase item difficulty. The presence of formulas clearly increased item difficulty, as it requires technical comprehension. Both UI elements and especially the possibility to interact with the system were found to decrease item difficulty. These kinds of items simulate real-life context, and at best, allow test-takers to try and observe the results of actions instead of leaning on memorising. The traditional text-based multiple-choice questions were also found to decrease item difficulty. These types of questions seem to require rather straightforward comprehension, and therefore they are not as challenging for the test-takers.

Keywords:

ICT skills, measurement, item analysis, item characteristics.