Abstract:

User evaluations on virtual museums (VMs) projects are planned because of the necessity to conduct test-beds and user experience surveys to have feedbacks and improve criticisms in the interactive software applications.

It is therefore a priority to test the impact that virtual applications in real museums produce on the audience: the role of storytelling, the clarity of contents, the emotional aspects of communication, the interaction mode, the rhythms of narration, the embodiment, the soundscape, the interface usability, definitively the design of the whole experience in the performative space of the museum. The more we are able to create an optimal convergence among the different media towards an expressive unity, the more the Virtual Museum will result in an effective educational vehicle.

From a methodological point of view, no specific direction has been taken both in humanities or computer sciences fields, but several and different strategies are employed whenever we deal with VMs, given the diversity and complexity of each of them, the target, the context of use and so on. Nevertheless, common issues have been highlighted in the past years, foreseeing the real possibility to draft a taxonomy for VM evaluations and an almost exhaustive methodology.

CNR ITABC has progressively developed a methodological approach for the user experience evaluation, focusing its attention on the pedagogical aspects and cognitive affordances. The massive work carried on under the V-Must network and other european projects as Etruscanning, or exhibitions like Digital Heritage Expo and Archeovirtual, have allowed to test several evaluation tools and theoretical procedures.

What is relevant here is the centrality of the end-user who need to be the main protagonist of any cultural public transmission and thus of any investigation programme.

Comparing methodologies used for the latest CNR evaluations, especially in the case of the “Virtual Museum of the Tiber Valley”, and “Livia’s Villa Reloaded”, in this contribute we come up with solid considerations: the relevance of the combination of quantitative and qualitative tools; the importance of observations as preliminary mean to study the public’s behaviors; the practicability of submitting driven-scenarios to analyze user’s control over the system; the distinction between active and passive users to better understand the fruition’s dynamics.

During the evaluation of the user experience in the museum, the role of operator is indeed fundamental: no emotions need to interfere with the judgments about the user experience; no suggestions or compliance need to be undertaken in driven-scenarios so to not influence people’s interaction. Language is another issue to care about: according to the final target, the support of evaluation’s delivery, the context of submission and the time at disposal, questions and “exercises” in the evaluation schedule have to be carefully planned to build an increase line of difficulty. All these considerations are part of a bigger apparatus of best practices we have drawn down. Much more can be done in this field of work, diversifying the typologies of surveys and improving the relative tools and strategies. For sure the multi-partitioned analysis, tested up to now - made of observation, questionnaire and driven-scenario - proved to be very functional and efficient as evaluative method, granting an overview on user’s behaviour in complex conditions of fruition

Keywords:

On site Virtual Museums, User experience survey, evaluation methodology, quantitative and qualitative analyses, cultural transmission, education.