Abstract:

The purpose of systems for digital competence (DC) acquisition is to help students with their teachers' support to acquire a set of DC that can be afterward easily recognized in the higher-level education or labor market. Although many studies examine the impact of such an approach on students, it is also essential to analyze teachers' perceptions as they are considered the primary agents of school change. The literature review did not identify any examples of measuring the long-term sustainability of DC acquisition and evaluation systems in education. Therefore, the main aim of this study is to determine the extent to which usability contributes to teachers' continuance intention to use the system for the acquisition and evaluation of DC within primary and secondary education.

For that purpose, the usability-continuance intention model, which emphasizes the need to examine the usability of educational systems and their relation with expected use in the future, was proposed. Based on the model, a survey instrument was created, consisting of six items on effectiveness, seven items on efficiency, four items on satisfaction, and three on continuance intention. It was received 353 survey responses in total in the period between 29 April and 30 June 2019 from teachers from six European countries (Croatia, Greece, Italy, Romania, Spain, and Sweden) who voluntarily used the system for acquisition and evaluation of DC developed as part of a Horizon 2020 project CRISS. There were no ineligible respondents.

We analyzed the proposed research model with PLS-SEM following the two-step approach:
(1) validation of the measurement model and
(2) examination of the structural model.

Research results exposed several important findings:
(1) Satisfaction with the system is the strongest predictor of teachers' continuance intention, as shown by other studies;
(2) The teachers' perception of efficiency (invested time, mental or physical effort) in using the system is more important in predicting their continuance intention than the perception of effectiveness (accurate and complete achievement of specific goals through the system), which showed to be non-significant;
(3) Satisfaction mediates the effects of both effectiveness and efficiency on continuance intention.

Overall, the continuance intention was directly driven by teachers' satisfaction level and efficiency perception and indirectly through satisfaction both by a perception of effectiveness and, once again, efficiency.

This study contributes to knowledge by studying user behavior after the information system adoption and examining the usability of systems that support teaching and learning processes. However, besides using quantitative instruments, in-depth interviews can also be employed to further research such systems. Also, the system evaluated in this study was one of the first attempts in Europe to formally implement DC in the educational process in primary and secondary schools, and teachers who participated in the project were the only ones who could provide relevant information. They were an abundant data source, but at the same time, this non-probability sample of teachers limited the generalizability of the study findings. Therefore, future studies could examine the same or other usability variables in similar educational contexts or DC systems.

Keywords:

Usability, continuance intention, primary and secondary education, PLS-SEM.