Abstract:

Future education and specifically higher and continuous education goes, no doubt, through e-learning and related practices. But e-learning by itself cannot cope with this challenge. It requires quality to be properly used in an efficient way. In order to assure quality of e-learning several ways have been set up, but ordinary e-learning is not the best method to take advantage of the computer for educational purposes.
With the arrival of Distributed Artificial Intelligence e-learning systems have increased quite a bit their capability and functionality by incorporating several techniques becoming intelligent e-learning systems. Now the agents supporting the system can learn form their experience by using machine learning techniques, can understand natural language such as English or Spanish, can evaluate the student learning by using fuzzy logic techniques, can analyze deep and shallow student errors, can have the experience of a human expert, and even can have emotions and understand also the student emotions in order to use affective and cognitive tactics to help him/her and ameliorate the human learning process. However, intelligent e-learning systems are still young to have standards. In spite of this, quality assessment is a “must” for those complex intelligent e-learning systems.
The paper analyses first of all the functionality of intelligent e-learning systems and the Artificial Intelligence techniques included from the user’s point of view, to set up possible standards and their limitations, taking into account the different ways of analysing and designing intelligent e-learning systems. Then the paper explores the few partial attempts so far done to assess the quality of those systems their pros and cons. APA, AERA and NCME published the 1999 Standards for Educational and Psychological Testing founded on the concepts of validity and reliability. They are also analysed although do not yet include appropriate methods for the evaluation of performance assessment in complex environments.
One of the important contributions of the paper is the development of a quality assessment methodology with two different parts. The first one is devoted to assess the functional quality of the system from the technical and the user’s point of view. It considers the features of the student’s model adopted by the system, the nature and implementation of the learning domain and human expert experience handled by the system, the techniques used for human error analysis (deep and shallow), the tutorial activities and their adequacy from the point of view of General and Specific Didactics, and the human/machine interface.
The second part of the methodology deals with the overall quality assessment of the system, splitted up in assessing the overall functionality of the system, its reliability, evidential and consequential validity.
The methodology is finally applied to the quality assessment of an intelligent e-learning system already developed to show practically how the concepts introduced in the methodology and methods of measurement are applied in a practical way.

Keywords:

Quality assessment, e-learning, intelligent e-learning systems, evaluation, multi-agent systems, artificial intelligence.