Abstract:

The paper presents three-year application results of a designed theoretical model for Educational Software Design and Development (ESDD) in teaching undergraduate students at the Faculty of Mathematics and Informatics (FMI) of Sofia University “St. Kliment Ohridski” (SU). The model is used in the elective course “Design, Development and Evaluation of Educational Software”, taught to students in Computer Science BSc programmes. The paper is third one of a series of three publications that observe the theoretical building, implementation, and use in academic teaching practice of an ESDD model, designed specifically for the needs of a particular group of students.

The first paper describes mainly the designed ESDD model for the needs of the particular target audience – Computer Science students, attending a course on ESDD. The second paper describes the teaching practice for the above course by applying the designed ESDD model.

The current, third, paper presents evaluation of the developed ESDD model by assessing the students’ course products (interim and final), collected from the three years of course implementation.

Four groups of evaluation criteria are used in the analysis of the results:
1. Criteria based on the structure, steps, and content of the ESDD model.
These criteria assess the adherence of the students’ interim and final products to the stages (and key activities in each stage) of the ESDD model.
2. Content criteria, based on the key content elements, taught in the individual course themes. These criteria refer to the individual key elements/qualities of the design and development of the software product, as taught in the course. They include, for example, key criteria related to the storyboarding, graphic design, interaction, navigation, etc.
3. Criteria about the quality of the supporting project and product documentation.
These criteria include the key parts/elements/qualities that have to be developed in the supported students’ project and product documentation.
4. Criteria based on the quality of the final product. The evaluation of the final products (working prototypes of educational software) is made against widely accepted criteria for educational software quality. In pursuing this, a short review of related up-to-date research and practice work about key educational software evaluation criteria is made – by groups and individual criteria. The aim is to elicit a subset of key criteria, needed for the evaluation of the final students’ products, having in mind the course environmental factors (type of students and purpose of the course).

Three year categories are formed, based on the yearly results from the three consecutive academic years of course implementation (2013-14, 2014-15, 2015-16). Averages, based on the total results of all three years are also calculated as references to individual years’ results, and as a summary of the course outcomes in general. The individual years’ results are analysed in relation to the development state and improvements of the ESDD model during each year. This approach helps to understand the appropriateness of the implemented adjustments in the ESDD model. A comparative final evaluation is constructed by combining the results of the four groups of evaluation criteria – both by individual years, and in total.

Main conclusions and further improvement steps are outlined about the qualities of the designed ESDD model and its application in teaching.

Keywords:

Evaluation by results, educational software design and development, design model, teaching, higher education course.