Abstract:

This study presents preliminary results of a meta-analysis that examines personalized adaptive learning (PAL) in undergraduate mathematics, identifying where, when, and for whom PAL is helpful in relation to mathematics outcomes.

Personalized adaptive learning (PAL) implements intelligent learning systems, integrates learner preferences, and analyzes individual learning data to create a unique learner path personalized to the needs of students. PAL has been increasingly adopted in the U.S. and may be particularly beneficial to students in mathematics such as college algebra and calculus as they serve as gatekeeper courses, especially for those majoring in STEM fields. However, PAL within undergraduate students with a focus on mathematics outcomes has yet to be fully explored with meta-analytic methods.

This investigation addresses two pivotal research questions:
RQ1: What is the average effect of personalized adaptive learning on undergraduate mathematics outcomes broadly (study 1) and specifically on algebra (study 2), based on the empirical literature?
RQ2: To what extent is the effect of personalized adaptive learning on mathematics outcomes moderated by institutional-related factors, course-related factors, PAL-related factors, student-related factors, and study-related factors?

Thirteen databases, those most likely to index PAL studies, were searched. At the first stage, titles and abstracts were screened. Articles excluded clearly violated one or more inclusion criteria. The second stage of screening was full-text review. Only articles meeting all inclusion criteria are retained. Retained studies will have data extracted that will allow the results to be meta-analyzed. Multilevel meta-analysis is the data analytic procedure due to the expected hierarchical structure.

Results to date:
The search identified 12,734 studies and of those 2,162 duplicates were removed, resulting in 10,572 studies at the abstract and title screening phase, all of which have been double screened. Of the 10,572 studies, 10,303 have been excluded, and 269 were moved to full text eligibility screening. Of the 269 studies, 82 studies have been double screened for full text eligibility with 69 of those studies excluded and 13 studies included. Of the 13 studies included to date, three are journal articles and 10 are dissertations. All have been published since 2006 with over 50% published in 2014 or more recently.

This project seeks to advance Undergraduate STEM Education research by meta-analyzing studies related to implementation of innovative technological advancements in instruction and how PAL intervention relates to students’ success in mathematics—going beyond results from just one institution, one setting, one sample. As a work-in-progress, next steps include completing the second screening for the remaining full text eligibility and then extracting data from the included studies. Two rounds of the data extraction form have been piloted on a sample of the included studies with full-text studies independently reviewed by all coders and disagreements discussed. The research team will train on extracting data with one additional sample of included studies. The remaining studies will be screened by two coders, one of which will be either the PI or Co-PI; content coding less than exact agreement will be discussed to reach consensus.

Keywords:

Meta-analysis, personalized adaptive learning, undergraduate, mathematics.