Abstract:

Feedback is not only an effective means of promoting and consolidating learning but also a crucial factor in what concerns increasing students’ writing skills, knowledge, and strategies. Hence, the importance of using specific tools that provide immediate scoring and feedback on student writing, such as Automated Writing Evaluation (AWE).

Advantages such as a great capacity to process massive information, the accuracy of the scoring engine, individualized feedback on essays regarding the linguistic properties, technical quality, and organization, and also support for multiple essay submissions underline the importance of using this kind of tools within the educational system not only to accelerate the learning process but also to improve students’ abilities to absorb and improve their knowledge.

Despite these benefits, many instruments for automated writing feedback are still limited in functionality, and most of them are proprietary, thus potentially restraining their use due to the cost factors.

In the attempt to advance the research on the topic of AWE, a recent competition (Kaggle, 2022) had the objective to develop a solution to the challenge of automatically classifying augmentative and rhetorical elements in essays written by the gymnasium and high-school students. More specifically, the task was to predict the human annotations of the students’ writings which experts previously evaluated.

This article aims to present a machine learning methodology for identifying and classifying various discourse elements based on an ensemble of transformer-based models while highlighting the importance of incorporating AWE among the tools used in the evaluation process of essays. Also, in addition to analyzing essays, future directions for applying the evaluation framework to scientific articles in specific domains will be discussed.

Keywords:

Automated writing evaluation, machine learning, transformers, BERT.