Abstract:

In this paper, we analyze Augmented Reality (AR) in the context of a semiotic theory of new media. AR is an emerging technology allowing the creation of augmented environments, in which computer-generated digital content is seamlessly combined and overlapped into our perceptions of the real world. While examining the literature on AR, we realized that a large number of researches on AR mostly deal with the development of technical features of AR: what are the best hardware or software components to use to have the best AR experience, how to improve the functionality of these components, and so on. In our paper, we propose a different approach to AR: we use some aspects of the semiotic theory of new media to analyze the objects (the apps) created using the existing AR technology. That is, we consider the technical features of AR as the elements of a signifying system. When combined according to sets of rules defined within the system, these elements can be exploited to create or develop new forms of communication.

In the first part of the paper, we focus on outlining the theoretical framework for our analysis. We adopt a perspective defined in the theory of new media, already hinted by MacIntyre, Bolter, and others [1] and we also use theories and tools of the semiotics of new media. Particularly, the use of some theories of text semiotics allows us to consider each object produced by any medium as a particular form of multimedia text. This perspective requires an investigation of the concepts of text, media, and multimedia, to find out a non-ambiguous definition of multimedia text that can be used to identify and handle all types of objects produced by any of the existing media or by a combination of media.

Once the theoretical framework has been defined, we examine more in detail one of the new existing media: augmented reality. Here, we intend to systematize the knowledge on AR from a semiotics of new media perspective, particularly with regard to the possible multimedia texts that can be created by using AR as a signifying system. Dealing with AR, these multimedia texts are the applications running on head-mounted devices capable of generating immersive augmented environments.

Finally, as an example on how to use the theoretical framework outlined above, we analyze a particular app we developed, capable of generating a learning environment in AR for older adults. The app, tailored to older adults, is easily accessible without any previous knowledge of technology, and capable of delivering educational materials on digital culture. During the analysis of this app, we treat the app as a particular form of multimedia text within the domain of education, developed by using augmented reality as a signifying system.

The methodology of analysis proposed allows seeing in an analytical and systematic way the objects produced with AR technology and can be used in different contexts for the analysis of a variety of existing AR apps. Moreover, the methodology of analysis might prove a valuable helping tool during the design of new apps, as it increases the awareness on the existence of structures, features, and behaviors of the apps to be developed.

References:
[1] B. MacIntyre, J. D. Bolter, E. Moreno, and B. Hannigan, “Augmented reality as a new media experience,” in Proceedings - IEEE and ACM International Symposium on Augmented Reality, ISAR 2001, pp. 197–206.

Keywords:

Augmented reality, e-learning in augmented reality, semiotics of new media, new media theory.