Abstract:

Development of an effective automated knowledge control system is necessarily connected with implementation of software module for grading of the control test answers, which are freely formulated in natural language. Previously, we presented an experimental prototype of such a system and conducted an experiment in which short answers of basic types received from students were processed by pragmatically oriented question-answer text processing algorithm in order to evaluate its output and algorithm flaws. Based on these experimental results, problematic situations in algorithm functioning were identified and ways of their solution were applied. In general, the obtained results show that chosen pragmatically oriented approach allows to solve the answer grading problem for basic questions involving explanation of typed or compound relations. However, there are questions with answer of more complex type, which require to describe several typed or compound relations around question object. These are questions of “Definition” and “Description” types, an experimental prototype for automatic answer grading on them is developed and presented in this paper. We discuss application of the same pragmatically oriented approach to this problem, which involve, as the main idea, segmentation of complex question-answer texts into basic parts containing already covered answer structure variants. An experiment on implemented automatic answer grading prototype was also conducted, which involved authentic knowledge control in university, students’ answers were processed by algorithm, results and conclusions are also presented in this paper.

Keywords:

Natural language processing, automatic answer grading, e-Assessment, automated assessment.