Abstract:

Museums are important to education because they facilitate interaction between visitors and exhibits. To improve the educational impact on museum visitors, it is necessary to understand their experiences. To this end, previous studies have used a wearable eye tracker. Wearable eye trackers measure the direction of an individual’s gaze, informing us of what they look at. These data provide insight on which exhibits are more visually appealing and, conversely, which ones often go unseen. By understanding these dynamics, museums can make their exhibits more effective. However, this measurement method has some drawbacks. Although it is possible to use a wearable tracker to accurately measure the subjects’ gaze direction in laboratories, it is impossible to measure visitors’ gaze direction in real museums. First, it is impractical to lend every visitor an eye tracker and teach them how to use it. Second, visitors may also jostle the eye tracker and upset its alignment, which prevents proper measurement and understanding of their experience. Finally, having the eye tracker on may restrict visitors’ movement and adversely affect their learning.

To solve this problem, the authors propose a method of eye measurement and gaze estimation that uses non-wearable sensors. This method uses an RGB-D camera as a non-wearable sensor: a person is detected, and their position is measured by the sensor. The eyes are detected using an image of the face, and gaze direction is subsequently estimated using the eye images. With a combination of body position and gaze direction, we can infer where the visitor is looking within a 3D space. Since the purpose is, of course, to estimate what visitors are observing, objects are detected simultaneously. To this end, the RGB-D camera employed as a non-wearable sensor is not only used to estimate gaze direction but also for object detection. Using the RGB-D camera, we can detect the object and estimate its position. By measuring the visitor’s position and gaze direction, as well as the object’s position, we can determine the relationship between the former and the latter, thereby estimating what the visitor is looking at. Preliminary experiments support the feasibility of this proposed method.

Keywords:

Gazed object estimation, Kinect sensor.