Abstract:

Since the project "Augmented Reality Sandbox" (2014) the AR Sand Table is a method widely used for projecting information onto a sand surface. But only the relationship between sea level and color, or water flow direction, has been demonstrated at many museums. Even though the interaction is intuitive and satisfying, no help for understanding more complex interconnections between factors like geology, weather, climate and human interventions is provided.
Therefore, we intend to augment a sand table with software and interfaces to simulate complex processes like erosion and present the results in AR, VR, and FTVR and of course on the sand.
We present a work in progress, with interim results already available. As a final result of the project, software solutions, construction instructions, and didactic best practice examples will be developed to assist teachers in implementing such applications in the classroom. The results are going to be tested and evaluated in classroom settings.
The application's goal is to investigate the effects of human intervention on nature in an interactive manner. We created a simple version of a sand table using ordinary play sand, a depth camera, and a standard projector, and then expanded the functionality with a generative AI-model that was trained using DEM-data (Digital Elevation Map = height data) from different landscapes and the associated satellite photos. This model can now generate credible satellite images based on new height data created by analyzing the sand surface, taking into account not only height but also slope and direction of the hillside.
The technological solution is based on a Kinect camera and the Unity game engine. The Kinect's depth camera measures the height of the sand surface and passes the data to a Python application, which uses the appropriate AI-model and returns the newly generated Satellite image to the Unity application, which projects it on the Sand Table. Additionally, several simulations can be applied to the surface data, e.g., eroding the surface, changing the vegetation depending on availability and amount of water and the development and destruction of moorlands.
Using this approach in classrooms allows not only to discuss the connection between sea level, vegetation zones, influence of slope, direction of hillside on how a landscape looks like but also the possibilities and restrictions of generative AI, Machine Learning and pattern recognition.

Keywords:

Geoscience, Data Visualization, Augmented Reality, education, augmented reality sandbox.