Abstract:

Large amounts of funds are spent every year to implement social and educational interventions in developing countries. Realizing such educational interventions is a complex task spread out over months or years, and involves negotiating a variety of logistic issues while relying on various stakeholders to play their stipulated roles. Intervention fidelity is a measure of how faithful an educational intervention is in implementing the processes prescribed in the program. A number of studies have shown that better “fidelity” leads to stronger effects (Nelson, Cordray, Hulleman, Darrow, & Sommer, 2012) which simply means that interventions with better fidelity show statistically superior results to those that do not. A framework for fidelity for educational interventions called FOI has been proposed (Century, Rudnick, & Freeman, 2010). FOI suggests two primary critical components of adherence; Structural Critical Components (SCC) and Instructional Critical Components (ICC). The SCC is, in turn, divided into Procedure and Educative dimensions. Two aspects of Procedure vital for educational interventions are Exposure and Dosage. Dosage is about “how much time is spent?” while Exposure addresses “in what way that time was spent?” The second sub-component of SCC is Educative and consists of capacity building of key stakeholders. ICC is divided into Pedagogical and Student Engagement. The Pedagogical component is about assessing how close the pedagogical delivery comes to an ideal or a benchmark of perfection. Student Engagement is the degree to which students are participating and engaged with the intervention. Finally, Differentiation is concerned with unique features of the intervention as distinguishable from other programs (including the control group). Mobile devices and Internet of Things (IoT) technologies including low-cost sensors and specialized networking protocols combined with big data analytics can be used to monitor educational intervention environments in a cost-effective manner. This paper considers experiences gained in implementing technology-based educational interventions in a developing country over a three year period. In these educational interventions, a variety of educational technologies including learning management systems, mobile devices and satellite-based virtual classrooms were used to train teachers, include parents and society, and to provide teachers one-on-one mentoring based on formative assessments and classroom observation videos. A total of over 350 public schools were involved. This paper will develop and present a use-case analysis of where and when fidelity was compromised during these three years. The paper will then present a classification of these violations based on FOI and explore their causes. The paper will end with providing future guidance on how each of these classes of fidelity violation can be potentially mediated via scalable and cost-effective use of mobile, IoT and big data analytics.

References:
[1] Century, J., Rudnick, M., & Freeman, C. (2010). A framework for measuring fidelity of implementation: A foundation for shared language and accumulation of knowledge. American Journal of Evaluation, 31(2), 199–218.
[2] Nelson, M. C., Cordray, D. S., Hulleman, C. S., Darrow, C. L., & Sommer, E. C. (2012). A procedure for assessing intervention fidelity in experiments testing educational and behavioral interventions. The Journal of Behavioral Health Services & Research, 39(4), 374–396.

Keywords:

Educational interventions, developing countries, fidelity, big data, analytics.