Abstract:

In this paper we discuss empirical evidence from extensive students’ evaluations of a Compendium Platform (CP) that was developed to be used in several undergraduate courses to support effective non-rote learning of statistics within a constructivist framework. (freestatistics.org) The CP, allowing students to archive, review and reproduce statistical calculations, has been improved and used for more than three years and it has been evaluated by more than 410 students in 2008 and 2009 (by means of several surveys ). Based on these data from several groups of undergraduate students, we are able to assess the CP and to explore various relationships between different e-learning success measures and external students’ learning attitudes.
In order to evaluate the success of an e-learning system such as the CP, several ICT acceptance and success models can be found in the literature. In previous research, we built and validated an integrated e-learning evaluation model, using the Delone & McLean IS Success model and Davis’ Technology Acceptance Model (TAM) (Poelmans et al., 2008). The structural explanatory model presupposes that multi-dimensional measures such as information and system quality, will impact success measures such as perceived relative advantage, ease of use, and the intention of a student to use the e-learning system in the future. In this study, the model is enhanced with the results of a learning attitudes’ survey (ATTLES), measuring whether students are “connected” or “separate” learners (Galotti et al., 1999). These two measures are introduced as external explanatory factors.

To generate a rich interpretation of the complexity of the e-learning environment, we turned to the social constructivist perspective and also measured students’ appraisal of the CP with the Constructivist On-Line Learning Environment Survey (COLLES). COLLES can be interpreted as an alternative students’ satisfaction survey with multiple dimensions such as: relevance, reflection, interactivity, educator & peers support and mutual understanding (Taylor et al., 2000).

In general, the results show that the CP has been well accepted, with a majority of the students giving positive scores (more than 3 on 5-point Likert scales) and mean scores on several success measures that vary between 3.5 and 4.

Using techniques like Partial Least Squares (PLS) regressions and correlations, we can validate the structural evaluation model and show significant relationships between most of the constructs. Specific measures such as information and system quality are reliable success predictors that can provide designers of e-learning systems with specific guidelines. Further results show that students‘ learning attitudes only have a limited effect on a few success measures.

Finally, most of the dimensions of the COLLES survey are significantly correlated with several measures of our evaluation model, confirming the validity of the model. Although COLLES scores can be used as predictors of ‘intention to use’ and other success measures ; their explanatory power is not very strong. Given the broader, pedagogical scope of COLLES, this result is not surprising.

Keywords:

e-learning, virtual learning environment, information systems success, evaluation.