Abstract:

Although Khan Academy (KA), with its 60 million users per month, is one of the world’s most popular education websites, there is a lack of research on its impact on students’ skills. In this paper, we will present our action research that investigated students’ mathematical skills before and after KA homework practice.

Introduction:
Research has shown that there is a significant correlation between homework and achievement in math from grades 7 to 12. Other studies have shown that immediate feedback and acknowledgement of homework effort is important for students progress. Appropriately difficult and challenging homework for individual students is ideal. KA offers free interactive math exercises that allow teachers to monitor students’ activities and to individualise their homework assignments. KA’s exercises provide students with guidance and immediate feedback.

Research question:
We have been implementing and evaluating KA as a high school math homework platform for over two years. Early on, we were surprised that some students were unable to solve analogical problems, whereas they had successfully solved similar KA homework problems previously. We therefore conducted a study to answer the following question: Are students able to transfer skills acquired from KA to other contexts?

Methodology:
Czech students (N = 44) from two 11th grade classes were tested once a week for 17 weeks, which produced n = 604 observations of their abilities. Local conditions did not allow us to conduct a classical treatment-group/control-group study. Students were divided into four groups of 11 students, using a randomized block design based on their math grades so that each group had a similar distribution of math grades. Each week, the students were required to practise their math skills by using KA at home. Before the practice, the first group was tested in the KA context and the second group was tested in a different context similar to a traditional Czech textbook. After the practice, the third group was tested in the KA context and the fourth group was tested in the different context. Each week, the groups switched roles so that, over the course of 17 weeks, each student was tested in each setting several times. The collected data were subject to null hypothesis testing with a significance level of ⍺ = 0.05.

Results:
Overall, students’ test results improved significantly after KA practice, but they improved significantly more in the KA context. Even though most students had better results after practice in both contexts, a large number were regularly unable to solve problems analogical to KA problems in the different context after the practice.

Limitations and future research:
As our student sample was from one urban high school in Prague, Czech Republic, the results are not directly generalizable. The factors that influence student abilities to transfer skills acquired from KA to other contexts have yet to be verified. This will be one of our upcoming research questions.

References:
[1] Fan, H., et al. (2017). Homework and students' achievement in math and science: A 30-year meta-analysis, 1986–2015. Educational Research Review, 20, pp.35-54.
[2] Attali, Y. (2015). Effects of multiple-try feedback and question type during mathematics problem solving on performance in similar problems. Computers & Education, 86, pp.260-267.
[3] Murphy, R., et al. (2014). Research on the Use of Khan Academy in Schools. Menlo Park, CA: SRI Education.

Keywords:

Homework, Interactive exercise, Khan Academy, Skill transfer.