Abstract:

Educational technologies are increasingly being used to address access, quality and governance issues. However, in many instances, these interventions are funded by donor agencies with specific demographic preferences and priorities. These preferences and lack of adequate funding often makes randomized trials and subsequent analyses unfeasible in these situations. Consequently, the analysis in such scenarios is often reduced to consider qualitative aspects only or based on limited sample-based statistical techniques. Consequently, the inherent complexity of implementation contexts in developing countries makes the problem of making meaningful refinement of the underlying theories of change difficult. A theory of change defines the building blocks to bring about a given long-term change in learning-related outcomes like access, quality or governance. A theory of change also articulates the pathways of change required to bring about a desired change in learning-related outcomes. For example, a theory of change can stipulate that using real-time student performance data to enhance feedback to teachers and to provide just-in-time focused continuous professional development will change teachers’ classroom behavior, which in turn will result in better short-term learning gains for students. A theory of change also articulates the assumptions and expectations about how and why proposed interventions will bring them about. Typical educational interventions generate a large amount of rich data that may consist of longitudinal data of student and teacher performance, qualitative data about student, teacher and educational administrators self-reported psychological (e.g., engagement, affect) or social profiles, and demographic data including socio-economic backgrounds of the various stakeholders. Educational analytics techniques have been used predict student academic performance in schools. Similarly, neural networks, support vector machine (SVM), decision trees, and multinomial logistic regression have been used to predict and analyze secondary education placement-test scores. Similarly, data-driven discovery to construct better student models to improve student learning have been proposed and contextualized, differential sequence mining method have been used to derive students' learning behavior patterns. This paper uses data collected over three years from donor-funded technology-based educational interventions in a developing country, and applies a variety of educational analytics techniques to show how use of such techniques can refine the various components of the underlying theory of change.

Keywords:

Educational analytics, theory of change, developing country, technology-enhanced learning.