DIGITAL LIBRARY
RELIABILITY EVALUATION OF KNOWLEDGE E-TESTING BASED ON STATISTICAL METHODS
1 MSTU "STANKIN" (RUSSIAN FEDERATION)
2 Federal State-Funded Institute of Science Institute for Design Problems in Microelectronics of Russian Academy of Sciences (RUSSIAN FEDERATION)
3 Gazprom Corporate Institute (RUSSIAN FEDERATION)
About this paper:
Appears in: EDULEARN17 Proceedings
Publication year: 2017
Pages: 2953-2962
ISBN: 978-84-697-3777-4
ISSN: 2340-1117
doi: 10.21125/edulearn.2017.1616
Conference name: 9th International Conference on Education and New Learning Technologies
Dates: 3-5 July, 2017
Location: Barcelona, Spain
Abstract:
Further ICT development and accessibility causes heightened interest in the concept of educational process efficiency considered to be a guarantee of successful course within deadlines with less time losses and without decreasing retention level.
The key problem of the successful ICT application is a necessity to formalize teacher-learner interaction. It is especially important for getting proper answers to the eternal questions: how is a teacher aware of the trainees’ knowledge level and what is their readiness to continue their course training?
So, our main objectives are creating a system of electronic tests and introducing them into teaching practice under standard principles and recommendations, using huge reserves for increasing training effectiveness found in the development and application of special kind of tests – training ones, making possible arranging vocational training in small groups, while creating individual learning paths in heterogeneous groups, as well as training in groups with a non-uniform schedule.
Proposed training tests specificity is explicit and implicit prompts in their tasks, wording and distractors. In addition, these tests can be well applied not only for assessing the current level of knowledge, but also for their rapid recovery and systematization, which allows shortening classroom instruction preparatory stage to a great extent.
It is very important to mention that usual practices of creation and delivery of different assessment tools and services, including e-tests are focusing on high-stake ones, to be evaluated by formal procedures to confirm that they are reliable, valid, recognizable and realistic. As a rule, very extensive focus groups and long-term statistical test results are used in these procedures.
Unfortunately, such practices are useless in small groups – math statistics laws used in big groups have limited application in small ones because of non-representative samples. Therefore, some simplified methods for evaluation e-tests quality are needed, i.e. with appropriate approaches to forming focus groups and special statistical methods applicable to small samples.
The article presents functional models for developing and applying training tests in the learning process as well as the results of the proposed methods approbation for quality assurance and evaluating reliability of knowledge e-testing for undergraduates and graduate students in small groups (5-10 people). Evolution and adaptation of e-tests to the tasks of a separate educational trajectory are carried out by evaluating e-tests results statistically using mathematical methods adapted for small samples.
The practice of applying these statistical methods makes it possible to judge whether the training tests carried out in the proposed facilitated format remain reliable. They also remain valid when used to set minimum scores, i.e. when the results of these tests serve as a formal boundary between the "unsatisfactory" and "satisfactory" assessments (grades). In addition, these statistical methods can be further simplified and successfully applied in intelligent learning systems with feedback using the Learning Analytics concept.

Contents:
Introduction
1) Educational process efficiency concept
2) Various test types and their roles in the educational process
3) Training tests and tasks composition features
4) Assessment of educational process efficiency
5) Training tests validation problems
Conclusion
Acknowledgements
References
Keywords:
education, information and communication technology (ICT), statistical methods, e-learning, e-tests, small groups