Abstract:

Students in data science programmes often struggle with interpreting cluster visualisations such as k-means outputs, DBSCAN plots, and violin-style distribution summaries. Recent work on our chart-to-text assessment pipeline has shown that it is possible to generate verified true/false items and concise, chart-grounded narratives using a locally hosted multimodal model, with evidence of strong perceived learning gains and higher confidence in an authentic classroom deployment. Building on that technical study, this paper shifts the focus from the pipeline itself to the design and evaluation of an AI-supported formative assessment tool, EduViz Narrator, as it is actually used in a Jupyter-based teaching environment.

EduViz follows an “answer first, explain later” pattern: students upload a clustering visualisation inside a notebook, respond to automatically generated true/false questions, and only then receive item-level feedback and a narrative explanation. The tool is implemented entirely within a Jupyter notebook and deployed offline using LLaVA via Ollama, with three explicit interaction principles: a single-focus screen (one image, one compact question pane, one submit action), immediate closure of the feedback loop, and low-friction setup on students’ own machines. We report on the functional and non-functional requirements that guided the design, and on an extensive programme of unit and system testing (prompt construction, answer validation, narrative generation, stress and edge-case tests) conducted prior to classroom use. These tests demonstrate that an offline multimodal setup can remain robust and responsive under realistic teaching conditions, handling repeated uploads, multiple student machines, and corrupted or oversized images without kernel resets.

Using data from the same in-class deployment as the earlier pipeline study, we complement previously reported learning outcomes with a deeper analysis of interaction patterns and student perceptions of the Jupyter-based workflow. Questionnaire responses and open-text comments indicate that students experienced the notebook integration as simple and cognitively focused, particularly valuing the immediate, image-grounded quiz feedback, with narratives supporting subsequent reflection. At the same time, persistent difficulties with certain statistical concepts point to the need for additional scaffolds beyond automated feedback. We distil from these findings a set of design principles for “answer-first” AI support in data science education—offline-by-design deployment, verified chart grounding, and tightly integrated notebook interfaces—and discuss how they can inform future AI-supported formative assessment tools in STEM curricula.

Keywords:

Formative assessment, data science education, cluster visualisation, artificial intelligence in education, Jupyter notebooks, offline AI tools.