Funding Entity: 

This actuation has been subject to funding inside the budget of the Spanish Ministerio de Economía, Industria y Competitividad, with number RTC-2016-5295-7

Start Date: 
Thursday, 1 September 2016 to Friday, 21 December 2018
Hybrid Visualization System for Virtual and Augmented Reality with high resolution and compactness

The objective of the Project is to design and manufacture a first prototype of a hybrid visualization system for Virtual and Augmented Reality. It will have large field of view, high resolution, outstanding compactness and excellent ergonomics.

The technology of project SHIVA is focused on the Near-To-Eye (NTE) projection systems. Traditionally, those systems consisted of one or more displays together with an imaging optical system that will image on a virtual screen that will be visualised by the user. The components of the system are assembled in a head-mounted device (usually called head-mounted display). The system usually has an eye-tracking system and/or a head-tracking system, which information is used to render the image on the displays according to the head movements of the user, thus providing an immersive and 3D experience.

Nowadays there are two main types of NTE visualization systems:  Virtual Reality (VR) and Augmented Reality (AR). In VR, the user only observes the information depicted on the displays, whereas in AR, that information is overlapped on the environment that surrounds the user.

At this moment, VR and AR systems are experiencing an impressive growth, due to technical improvements and cost reductions of processors, movement sensors and high resolution displays. Several commercial products are reaching the mass consumer market.

In a near future, it would be ideal the convergence of VR and AR systems, towards a hybrid VR and AR system, able to work in both modes. Consequently, the isolation from the environment caused by VR could be avoided by switching to the AR mode. And the large field of view provided by VR systems could improve significantly the possibility to show enhanced information in AR.

However, nowadays both AR and VR systems have very different features that hamper their convergence:

  • Immersive VR systems need a large field of view (FOV, over 90º), but the optics used to achieve that FOV result into bulky and non-ergonomic systems. This fact hinders their use for AR, where users usually interact with other users in a social environment.
  • Current VR systems have low latency, but they require an external element in a fixed position compared to the ground to perform not only the angular tracking of the head, but also the absolute position of the user. Due to this fact, the VR systems are not portable (or require a tracking system covering all the room), thus making them incompatible with AR applications.
  • The compact AR systems have a reduced field of view or they are monocular (no 3D), so they cannot be used for immersive and stereoscopic VR.
  • AR systems perform a tracking of objects and markers (for Mixed Reality applications) with a large latency, being incompatible with the positional head-tracking systems of VR.

The main specific objectives of project SHIVA are:

  • Design and manufacturing of high-coompact freeform multichannel optics with total internal reflection (thickness<20mm), high resolution (>11 pixels/degree) and large field of view (>100º)
  • Development of a low latency tracker (<10 ms), using displays at >70 frames per second rates, using prediction and time-warping techniques and advanced display addressing strategies.
  • Feasibility study to achieve a positional tracker with precision < 10 mm, using inertial sensor fusion and camera image processing.
  • Design and development of bidirectional video control electronics for VR and AR.
  • Development of a calibration system to achieve mapping algorithms for VR and AR.
  • Development of demo applications for VR, AR and MR, including the user interface.