gb2sμMOD: A MUltiMODal biometric video database using visible and IR light

Abstract: 

Recent studies encourage the development of sensorially-enriched media to enhance the user experience by stimulating senses other than sight and hearing. Sensory effects as odor, wind, vibration and light effects, as well as an enhanced audio quality, have been found to favour media enjoyment and to have a positive influence on the sense of Presence and on the perceived quality, relevance and reality of a multimedia experience. In particular, sports is among the genres that could benefit the most from these solutions. Several works have demonstrated also the technical feasibility of implementing and deploying end-to-end solutions integrating sensory effects into a legacy system. Thus, multi-sensorial media emerges as a mean to deliver a new form of immersive experiences to the mass market in a non-disruptive manner. However, many questions remain concerning issues as the sensory effects that can better complement a given audiovisual content or the best way in which to integrate and combine them to enhance the user experience of a target audience segment. The work presented in this paper aims to gain insight into the impact of binaural audio and sensory (light and olfactory) effects on the sports media experience, both at the overall level (average effect) and as a function of users’ characteristics (heterogeneous effects). To this aim, we conducted an experimental study exploring the influence of these immersive elements on the quality and Presence dimensions of the media experience. Along the quality dimension, we look for possible variations on the quality scores assigned to the overall media experience and to the media components content, image, audio and sensory effects. The potential impact on Presence is analyzed in terms of Spatial Presence and Engagement. The users’ characteristics considered encompass specific personal affective, cognitive and behavioral attributes. At the overall level we found that participants preferred binaural audio over standard stereo audio and that the presence of sensory effects increased significantly the level of Spatial Presence. Several heterogeneous effects were also revealed as a result of our experimental manipulations. Whereas binaural audio was found to have a generalized impact on the majority of the quality and Presence measures considered, the effects of sensory effects concentrate mainly on the Presence dimension. Personal characteristics explained most of the variation in the dependent variables, being individuals’ preferences in relation to the content, knowledge of involved technologies, tendency to emotional involvement and conscientiousness among the user variables with the most generalized influence. In particular, the former two features seem to present a conflict in the allocation of attentional resources towards the media content versus the technical features of the system, respectively. Additionally, football fans’ experience seems to be modulated by emotional processes whereas for not fans cognitive processes –and in particular those related to quality judgment– prevail.


Publication type: 
JRC-SCI Magazine
Published in: 
Information Fusion, pp.: 64-79
ISBN/ISSN: 
1566-2535
Publication date: 
November 2016
CeDInt Authors: 
Other Authors: 
Ignacio de Mendizábal-Vázquez