HOW TO MONITOR THE USE AND REUSE OF OPEN EDUCATIONAL RESOURCES USING GOOGLE ANALYTICS

Introduction
The “open education” agenda is changing attitudes within higher education in the UK and globally, and is increasing the amount of course materials being made freely available through the internet. The level of activity producing open content currently outweighs the attempts to understand whether and how it is being reused. The reuse of course materials such as learning objects has been defined as promoting “the availability of learning objects for others to use” (1). This definition assumes that available materials are then used in a meaningful way. The definition should be extended to include a level of behavioural change such as repurposing the material, or the repeated use of the material. Here lies the challenge – how to monitor reuse in terms of both availability (reach) but repurposing and repeated reuse (impact). The aim of this case study was to develop a methodology for tracking the reuse of an open-content website. The “Virtual Analytical Laboratory” or “VAL” is an open laboratory skills resource (2), part of the UK Centre for Bioscience project to produce a freely available laboratory and fieldwork manual (3).

Methods
User activity was tracked using Google Analytics, and an on-line survey tool (Survey Monkey) was employed gain feedback on user behaviour; both tools are freely available and automatically collect data, although in this study the survey tool was upgraded to extend the functionality.

Results
How to monitor reach? VAL consists of introductory pages and resource pages containing video and animation. Google Analytics data was analysed from the site launch in September 2008 to April 2010. The total number of visits (number of browser sessions) was 9,365 from 108 countries, predominately UK and USA. Looking at the “average time on site” provided an indication of the nature of the visit; visits that accessed VAL directly spent twice as long on the site compared to those who had found VAL on a search engine, suggesting that surfers were less serious users. How to monitor impact? The “visits” metric is an indicator of reach and initial use, but the “returning visitor” metric reflects reuse, and in total there were 1,729 returning visitors (19% of all traffic). Of these, 60% returned 3 times or more, providing an indicator of some degree of loyalty. Returning visitors tended to view the site for longer periods of time and viewed more pages. The attempt to evaluate the level of impact through the on-line survey yielded poor results with only 20 responses from users who were a mix of academics and non-academics, including an accountant and a trash collector.

In conclusion, the reuse of open educational content should be defined in terms of both the reach and impact. This study provides a methodology for monitoring the use and reuse of web-based open resources, although the tracking data requires careful interpretation. All these approaches require further development, and ultimately by monitoring the reach and impact of resources, academic institutions will be able to understand more fully the benefits and challenges of the open education agenda.

References:
(1) Koper R. (2003). In: Littlejohn A Reusing online resources. Oxford: Routledge. 46-59.
(2) Rolfe V. (2009). Available: http://www.bioscience.heacademy.ac.uk/ftp/casestudies/VRolfe.pdf. Last accessed 14 April 2010.
(3) UK Centre for Bioscience. (2010).
Available: http://www.bioscience.heacademy.ac.uk/resources/oer/. Last accessed 14 April 2010.