Abstract:

The paper presents three-year application results of a designed theoretical model for Educational Software Design and Development (ESDD) in teaching undergraduate students at the Faculty of Mathematics and Informatics (FMI) of Sofia University “St. Kliment Ohridski” (SU). The model is used in the elective course “Design, Development, and Evaluation of Educational Software”, taught to students in Computer Science BSc programmes.

The paper is third one of a series of three publications that discuss the theoretical building, implementation, and use in academic teaching practice of an ESDD model, designed specifically for the needs of a particular group of students. The first paper describes mainly the designed ESDD model for the needs of Computer Science students, attending a course on ESDD. The second paper describes the teaching practice for the above course by applying the designed ESDD model.

The main goal of the current, third, paper is to evaluate the developed ESDD model by assessing the developed students’ course products (interim and final), collected from the five years of course implementation.

Three groups of evaluation criteria are used in the analysis of the results:
1. Instructional Analysis quality criteria. These criteria assess the quality of the students’ analyses, performed for their chosen educational problems as required by the Analysis Phase of the ESDD model.
2. Design criteria, based on the design steps of the ESDD model, and on the key design principles and elements taught in the individual course themes. They include, for example, key criteria related to the planned software functionalities, activities, and tasks, the envisaged interface design, interaction, navigation, etc. – as developed in the storyboarding, and other respective interim assignments. A critical criterion is also the implementation of the analyses’ results in the software design.
3. Criteria based on the quality of the final product and the supporting software documentation.

The evaluation of the final products (working prototypes of educational software) is made against widely accepted criteria for educational software quality. In pursuing this, a short review of related up-to-date research and practice work about key educational software evaluation criteria is made – by groups and individual criteria. The aim is to elicit a subset of key criteria, needed for the evaluation of the final students’ products, having in mind the course environmental factors (type of students and purpose of the course).

Five yearly categories are formed, based on the yearly results from the five consecutive academic years of course implementation (2013-14 to 2017-18). Averages, based on the total results of all five years are also calculated as references to individual years’ results, and as a summary of the course outcomes in general. The individual years’ results are analysed in relation to the development state and improvements of the ESDD model during each year. This approach helps to understand the appropriateness of the implemented adjustments in the ESDD model. A comparative final evaluation is constructed by combining the results of the three groups of evaluation criteria – both by individual years, and in total.

Main conclusions and further improvement steps are outlined about the qualities of the designed ESDD model and its application in teaching.

Keywords:

Evaluation by results, educational software, design and development, design model, software evaluation, teaching, higher education, course.