Abstract:

The use of computers for education opens new possibilities for students and teachers. Web-based learning systems allow students to control their own learning rhythm. Students can connect to the course from anywhere and at anytime. However, many of the currently available systems base the evaluation of the student’s knowledge and competence on Multiple-Choice Questions or fill-in-the-blank exercises, which in the opinion of many educators, psychologists and pedagogues is not enough to fully assess the highest cognitive skills (Sigel, 1999). Therefore, the use of the computer to evaluate students’ knowledge and competences, has recently been extended with the possibility of automatically evaluating students’ free-text answers (Mason & Grove-Stephenson, 2002; Christie, 2003; Ishioka & Kameda, 2004).

The Will Tools are a set of web-based Blended Learning applications to support both teachers and students (Pérez-Marín et al. 2007). They consist of: Willow, a free-text scoring system able to provide immediate and complete feedback adapted to each particular student; Willov, a monitoring tool for the teacher; Willed, an authoring tool to introduce the information for the courses; and, Willoc, an account administration tool. These systems have been used since the 2005/2006 academic year in the Computer Science department of the Universidad Autónoma of Madrid. In this paper, we present the results of a survey carried out with 56 students and 7 teachers during the 2005/2006 and 2006/2007 academic years to find out what students and teachers think about web-based Blended Learning applications such as the Will Tools, and how they can be improved.

In particular, we have found out that students like the idea of having a system to support their review after class, and teachers like the idea of receiving a report indicating which concepts have been mostly understood by the class, and which ones still need more revision. Moreover, some features specifically required by teachers and students have been: collaborative management of the courses, automatic generation of courses from templates written in natural language, user-friendly interfaces, flexibility in the assessment, and tools for teachers and students to analyze the students’ progress in the courses and to generate individual and any subset of students’ reports.

In this paper, it is also explained how these improvements can be implemented in web-based Blended Learning applications as it has been done in the Will Tools.

References:
Christie, J. (2003), Automated essay marking for content - does it work?, in 'Proceedings of the 7th International Computer Assisted Assessment Conference'.

Ishioka, T. & Kameda, M. (2004), 'Automated Japanese Essay Scoring System: JESS', Proceedings of the 15th International Workshop on Database and Expert Systems Applications, 4-8.

Mason, O. & Grove-Stephenson, I. (2002), Automated free text marking with paperless school, in 'Proceedings of the 6th International Computer Assisted Assessment Conference'.

Pérez-Marín, D.; Pascual-Nieto, I.; Alfonseca, E.; Anguiano, E. & Rodríguez, P. (2007), A study on the impact of the use of an automatic and adaptive free-text assessment system during a university course, in 'Blended Learning', Prentice Hall, Pearson Education, 186-195.

Sigel, I., ed. (1999), Development of mental representations: Theories and Applications, Lawrence Erlbaum Associates, New Jersey, U.S.A.

Keywords:

computer science education, e-assessment, natural language processing, open.