Abstract:

Within the context of our newly developed technology for Reproducible Computing (RC) we developed a learning environment which supports a socially constructivist approach (Von Glasersfeld 1987) to statistics education trough the use of a series of tools which support Reproducible Statistical Computations and Peer Review (PR) activities (Wessa 2009).
The first part of this paper discusses the newly developed Peer Review components that were designed, tested, and implemented without disrupting the existing learning environments in which they were introduced (BlackBoard and Moodle). In particular, our approach towards design and development may serve as an illustration of the fact that new learning components can: be developed in any language/database; be operated in commercial (closed box) and open source environments; be hosted on servers which are not necessarily controlled by nor located near the institution’s ICT department; be executed in a secure environment which uses the accounts of the VLE; keep track of learning activities of individual students (and store them in a separate database); be used for the purpose of formative assessment and educational research.
Based on a panel review the paper lists some shortcomings that need to be addressed in the upcoming releases. These shortcomings are caused by the fact that: some students do not read or follow written or oral instructions; many students used multiple browser instances to perform the peer reviews; few students wished to be able to upload revised versions of their documents or reviews; some courses have a high dropout ratio which causes a variation in the average number of documents that can be reviewed over time. The paper also briefly discusses the various ways that can be employed to deal with these problems.
The second part of the paper provides some descriptive statistics about the usage and performance of the system. In addition, the costs that were associated to develop, test, and operate the PR system (during a full academic year) in two academic institutions are described and estimated. It is shown that costs can be kept relatively low due to the economies of scale of distributed computing, and the use of free and open software.
The third part of paper provides an overview of the technological and educational benefits that we have observed and studied so far (Poelmans et al. 2008/2009; Wessa 2009; Wessa & De Rycker 2010). The most prominent ones are related to PR and the underlying pedagogical theories – the aspects of formative assessment and social constructivism are discussed while the potential benefits for reviewees and reviewers are highlighted. Even though it is difficult to quantify the learning benefits of new learning systems, it is explained that our attempts to predict such learning outcomes have been fairly successful and have the potential to provide valuable insights that improve the effectiveness and efficiency of the learning environment. Finally, there are secondary benefits that may emerge in future developments. This is, for instance, explained in the paper of Wessa and Baesens (2009) which addresses the technological aspects of an innovative fraud detection system that discriminates between various types of fraud (plagiarism, free riding, peer-review cheating), and the pedagogical issues that result from its implementation.

Keywords:

Educational technology, peer review, social constructivism, software design, software implementation.