ANALYSIS OF TWO RANDOMIZED FIELD TRIALS TESTING EFFECTS OF AN ONLINE VOCABULARY PROGRAM ON VOCABULARY TEST SCORES
1 University of Minnesota (UNITED STATES)
2 Seward Inc. (UNITED STATES)
About this paper:
Conference name: 5th International Technology, Education and Development Conference
Dates: 7-9 March, 2011
Location: Valencia, Spain
Abstract:
Low vocabulary knowledge hinders reading achievement for many children, particularly those who are economically disadvantaged or second language learners. Use of computer technology has been posited by reading researchers as a potentially effective and efficient strategy for remediation of vocabulary deficits. Two randomized field trials were conducted to test the effects of a Web-based, remedial vocabulary program on vocabulary tests scores. This software program is designed to deliver performance-adaptive instruction on approximately 4000 high frequency words in written English, using instructional strategies shown to be effective in teaching vocabulary. This includes use of user-friendly definitions, repeated exposure to target words used in context, and game-like activities designed to enhance speed of lexical access. Students in grades 1-4 with low vocabulary knowledge were randomly assigned to receive either the online vocabulary intervention or business-as-usual instruction. In Study 1 (N = 43), treatment students received instruction on a stratified sample of 100 target words from the 4000 word corpus over a period of two weeks. The 40-word vocabulary measures used as pretests and posttests were comprised of words from the 100 words targeted in the intervention. Treatment students outperformed control students on the posttests by more than one standard deviation overall (Cohen’s d > 1.0). In Study 2 (N = 192), treatment students received instruction on an individualized profile of words from the 4000 word corpus over a period of approximately 14 weeks. No significant difference in vocabulary scores was found between treatment and control students at posttest, consistent with long-held assertions that standardized tests lack the sensitivity required to detect changes in vocabulary knowledge over typical experimental time frames. Word-level analyses showed that Study 2 students’ posttest performance was significantly better on words they had encountered during the intervention. Students were twice as likely to respond correctly on posttest items if they had encountered the word during the intervention. This suggests that students gained word knowledge from using the program but did not encounter enough words during the intervention to affect standardized tests scores. Approximately one percent of the words targeted during instruction were tested for at the posttest. Online data collected by the software program showed that treatment fidelity was inconsistent. Many students received only a small fraction of the scheduled online instruction. The challenges inherent in scaling up an intervention shown to be effective during pilot studies are discussed. Further research is needed to determine if a more sustained intervention with higher treatment fidelity using this computer-adaptive approach to vocabulary instruction can improve vocabulary scores and significantly affect reading achievement for students with low vocabulary knowledge. Keywords:
Vocabulary, online instruction, randomized field trial.