Abstract:

In the study, we focus on the assessment of second-language learners’ written production. Such assessments are typically carried out by trained raters using standardized criteria aligned with proficiency frameworks. However, it remains unclear to what extent these expert evaluations correspond to how learner texts are perceived by ordinary native speakers. In the paper, we investigate this relationship by examining whether texts assigned to higher proficiency levels by professional assessors are also perceived by non-specialist readers as linguistically stronger or more comprehensible, and whether lower-rated texts are perceived accordingly.

To address these aims, we conducted the analysis with students of a Czech grammar school (upper-secondary level). The participants evaluated short texts written by learners of Czech as a foreign language. All texts were taken from the MERLIN corpus, a learner corpus containing texts from standardized language examinations that are methodologically aligned with the Common European Framework of Reference for Languages (CEFR). For the purposes of this study, we focus on a single text type, invitation letters, to maintain consistency and avoid potential genre effects. Given the scope of the study, we limit our analysis to two proficiency levels: A2 and B1, which are well represented in the corpus.

The MERLIN corpus provides analytic ratings for several criteria: Overall CEFR rating, Grammatical accuracy, Orthography, Vocabulary range, Vocabulary control, Coherence/Cohesion and Sociolinguistic appropriateness. In our analysis, the students evaluated each text according to the same criteria and additionally assessed its overall comprehensibility. All ratings were given on a six-point scale corresponding numerically to the six CEFR levels (1–6). The students were informed about each criterion but did not receive specific scoring guidelines. They had no prior experience with assessing texts written by non-native speakers.

The collected ratings were subsequently compared with the professional evaluations available in the MERLIN corpus. The analysis focuses on the degree of agreement between student and expert assessments, differences across individual criteria, and the extent to which students distinguish between A2 and B1 texts. The results provide insight into how inexperienced evaluators perceive linguistic quality and communicative effectiveness in learner writing.

From a pedagogical perspective, the study demonstrates the potential of using authentic learner corpus data in secondary education. Integrating evaluation activities based on authentic learner texts can support the development of linguistic awareness, critical reading skills and understanding of language proficiency levels among students. At the same time, the comparison with expert ratings highlights which aspects of language performance are most difficult for inexperienced evaluators to judge. These findings may inform teaching practices in language education and contribute to the development of classroom activities that connect linguistic research resources with learner-centered assessment and reflection.

Keywords:

Learner writing assessment, second language acquisition, learner corpus research, language assessment literacy, discourse.