Abstract:

What does it mean to be an expert within a given field? This presentation gives an overview of a small-scale research study which explores this question through the prism of one very specific domain – listening test item writing in TESOL (Teaching English to Speakers of Other Languages). Considerable attention has been given by different researchers to outcomes - the ESOL tests themselves - but very little consideration has been given to process: how the tests are produced by item writers. This research uses frameworks frequently found in expertise studies (see, for example, Ericsson et al. 2006) and concurrent-verbalisation (think-aloud) data-gathering techniques (for example, Bowles, 2010) to illuminate processes involved in listening test item writing. It compares the way small groups of novice and experienced test writers undertake the same item-writing task. The findings indicate that, although expertise in test writing is highly individualised, a number of identifiable strategies and practices are associated with expert outcomes.

The presentation will summarise the ways in which the insights from this study have the potential to inform ESOL test-writer training and it will also make an introductory case for using concurrent verbalisation to shed light on the cognitive operations used by experts in many different domains.

Keywords:

ESOL test writing, test-writer training, think-aloud, verbal protocol analysis (VPA).