Abstract:

It has been shown that connecting Computer Science (CS) to other fields and including real-world projects in introductory CS classes increase student interest, engagement, and retention. Advanced computing concepts, such as machine learning, the internet of things virtual reality, and cybersecurity, are all around us, and including them even in secondary education would make for a more engaging curriculum. However, the scope of introductory courses is often confined to basic projects, reflecting the learners' initial level of knowledge.

NetsBlox is an educational block-based visual programming platform specifically designed to introduce students to more advanced programming concepts through a visual, user-friendly interface. NetsBlox enabled beginners to create projects that access online data sources and services such as Google Maps, scientific databases, machine learning services, and many others. Students can write networked applications like multi-player games and a shared whiteboard.

In this work, we introduce ARBlox which extends NetsBlox to support Augmented Reality (AR). AR is a view of the real world that is augmented by adding virtual information. ARBlox empowers novice students to create AR projects with nothing but a webcam. NetsBlox and ARBlox are designed to function seamlessly on standard school computers. ARBlox allows educators to assign projects that are approachable to novice learners, but impactful and applicable to real-world problems; It hides unnecessary complexity while exposing fundamental concepts behind AR.

ARBlox leverages lightweight machine-learning models to detect and track hand, face, body, and AR tags. ARBlox enables a wide range of project possibilities. ARBlox includes only a few new blocks that return the coordinates of the tracked object in real time. Initial projects can be scaffolded by providing additional custom blocks that process the coordinates to further ease the initial learning curve.

We have developed a set of projects to illustrate the broad range of possibilities ARBlox enables. Using the hand detector, we introduce a simple hand gesture classifier; The classifier uses relative landmark coordinates to differentiate between gestures. Employing the face detector, we showcase a face-augmenting application; a dynamic sprite is positioned onto key landmarks to augment the face (e.g., adding a beard). With the body detector, we present an AR outfit selector; an article of clothing is positioned onto the body using key landmark coordinates. Finally, utilizing AR tags, we create virtual musical instruments; By detecting when an AR tag is occluded, we are able to create simple instruments.

These examples demonstrate ARBlox's ability to provide projects that are both beginner-friendly and meaningful. Traditionally, creating AR projects requires extensive coding expertise. In contrast, these projects require a few code blocks (e.g., as little as 10 to 15 blocks), rendering them highly accessible to beginners. Additionally, they empower students to explore AR solutions, an active, engaging field of study. With ARBlox, educators are able to build curricula that foster impactful, yet approachable projects, effectively bridging the gap between beginner-friendly and meaningful projects. While ARBlox aims to make CS appealing to a wider group of students by making learning more engaging and relevant, further research is necessary to empirically validate its effectiveness in achieving this goal.

Keywords:

Computer science education, augmented reality, visual programming, web browser.