Abstract:

As the amount and variety of student data captured by university systems increase, so do opportunities for learning analytics (LA) in support of teaching and learning. However, although the technical methods underpinning LA are developing rapidly (e.g., machine learning), researchers have paid less attention to the needs or concerns of human stakeholders. Existing studies that investigate stakeholders’ perspectives tend to focus on either the technology (Sun et al., 2019) or on specific stakeholders’ views of student data privacy (Alnald & Sclater, 2017; Ifenchaler & Shumacher, 2016; Slade et al., 2019; Tsai, et al., 2020). Existing studies have used both surveys (Kollom et al., 2021; Whitelock-Wainwrite et al., 2020) and interviews (Ifenchaler & Shumacher, 2016; Sun et al., 2019) to investigate stakeholders’ expectations about LA technologies.

Our study contributes to the LA literature in two ways: we used a scenario-based approach (Rosson & Caroll, 2002) to engage participants in concrete LA-supported contexts, and we investigated multiple types of stakeholders’ perspectives within the same contexts. To implement our scenario-based approach, we presented hypothetical LA situations and interviewed four students and four faculty about the use and impacts of the envisioned LA. During the interviews, we first probed participants’ personal understanding of LA, then presented each hypothetical scenario, gathering reactions specific to the situation (e.g., a virtual assistant that draws on LA to provide personalized real-time assistance).

Our qualitative analysis of participants’ comments uncovered insights regarding attitudes towards the use of a virtual assistant (VA) as well as a predictive LA model in support of teaching and learning. Students generally feel positive about having a VA in their classes, while faculty emphasized that VAs can assist but not teach. Students also voiced concerns about a predictive LA model that goes too far (e.g., discouraging a low-achieving student from taking a challenging class). Our interviewees provided interesting perspectives on student data ownership, appropriate uses, and protection of student data, as well as the potential for LA applications of these data. For example, students feel little control over their data even though they trust their institution to use it for the right purposes; faculty feel a personal responsibility for appropriate use and protection of student data in their teaching and learning. Students and faculty were in strong agreement that students should be involved in the decision-making process about new LA applications at the institution.

In our discussion of the findings, we offer preliminary guidelines for the collection and storage of student data, as well as implications for the design of LA technology and applications. Example design implications include the appropriate use of VA, balancing between knowing the right amount of information, using the right LA for the right purpose with consideration of student data privacy, and transparency in the use of student data trumps all. In our future work, we plan to expand our investigation by involving more students, advisors, and administrators as stakeholders and by developing more specialized scenario probes that evoke more detailed reactions and concerns.

Keywords:

Learning Analytics, Student Data Privacy, Scenario-Based Design, Intelligent System, Attitude, Stakeholders’ Perceptions.