This actuation has been subject of funding from the budget of the Ministerio de Economía, Industria y Competitividad, with the file number TEC2014-56867-R
The objective of the project is the generation of new concepts of advanced optics for Virtual Reality Headsets, paying special attention to the optical problems related to the size, field of view and resolution of such headsets, thus solving their maing problems of image quality and ergonomics. To achieve those goals, the project will design three new freeform optical systems: 1) multi-channel with 2 to 9 channels, 2) multichannel with 30 to 120 channels and 3) Light-Field generation and cascaded displays. One of the optical systems will be manufactured and tested, and a full VR headset prototype will be developed using that optical system.
The general objectives of the project lead to 6 specific objectives:
O1. Development of fundamental concepts for the analysis and development of VR optics with Field of View (FOV) >100 degrees, and headset size < 70 cm3, including:
- Analysis of the representation of freeform surfaces with orthogonal polynomials in rectangular regions, tesselable with continuity of at least order 2.
- Study of refractive-diffractive freeform surfaces to correct the chromatic aberration in systems with high resolution (<2 arch minutes).
O2. Design and simulation of optics for freeform architectures with several channels and large aperture and FOV >100 degrees, and size of the headset < 70 cm3, including:
- Optimization of the number of channels (from 2 to 9) to the possible aspect ratios of the display.
- Development of the partial derivatives equations system to define the freeform design problem of each channel when the aperture of the lens is larger than human pupil.
- Calculation of the macro-profile of the array that optimizes the range of movement of the eye and the field of view.
- Adjustment of the resolution of the channels to the angular resolution of the human eye, to minimize rendering times.
O3. Design and simulation of the freeform optics for multichannel architecture creating a Light-Field with FOV>100 degrees and size of headset < 45 cm3, including:
- Optimization of number of channels (from 30 to 120) depending on the specifications of the system.
- Calculation with NTE algebra of the optimal Light-Field to correct accommodation and convergence.
- Calculation with NTE algebra of the optimal Light-Field to achieve super-resolution by pixel interlacing.
O4. Design and simulation of optics for cascaded displays architecture and FOV > 100 degrees, size of the headset < 45 cm3, including:
- Optimization of the number of displays (from 2 to 4) depending on the specifications.
- Calculation with NTE algebra of the display addressing algorithm to correct accommodation and convergence.
- Calculation with NTE algebra of the display addressing algorithm to achieve super-resolution (spatial and temporal), evaluating signal to noise ration of the compression achieved.
O5. Manufacturing and characterization in lab of one freeform optic for VR with FOV > 100 degrees, size of headset <70 cm2, including:
- Selection of most adequate materials and technologies, considering quality/cost balance.
- Measurement of optical surfaces, including profiles and roughness.
- Measurement of image quality (MTF), focal, FOV, and mapping function.
- Modeling and simulation of the prototype manufactured and comparison with designs.
O6. Integration of a first prototype of VR headset based on the freeform optics manufactured, including:
- Selection and purchasing of displays.
- Design and manufacturing of the electronics to control the displays and the head tracking system.
- Programming of the shaders for 3D rendering of virtual scenes and mapping between scene’s pixels and display’s pixels, according with the selected design.
- Functional measurement of the prototype and evaluation with subjective tests.
The project is working on 8 main activities, having obtained the following results so far:
Activity 1: Representation of freeform surfaces with orthogonal polynomials.
The design process of a freeform optical system is divided in three phases: first, a design close to the final objective is developed using SMS (Simultaneous Multiple Surfaces) method, resulting in a cloud of points of the surfaces. Second, that surface is approximated with a set of orthogonal polynomials. Finally, the coefficient of the polynomials are optimized using the commercial software CODE V. The optimization phase depends strongly on the initial solution and the type of polynomials used, affecting the time of resolution an even the achievement of a solution or not. For this phase, OPTIVAR has carried out an exhaustive study comparing the different families of polynomials: XY, Legendre, Zernike and Forbes, to select the most adequate. The results allow the optimization of results using the most appropriate polynomial family, depending on the shape of the aperture. Fig. 1 shows an example of the results obtained.
Fig.1. Example of the results obtained for each orthogonal polynomial family.
Activity 2: Design of systems with diffractive surfaces.
With the SMS method in its current state of development, it has been possible to design systems with up to 4 optical surfaces to perfect focus 4 different points. Theoretically, N points could be focused with N surfaces, although the complexity of the problem grows dramatically with N. The analysis show, as expected, that the more points are focused, the closer is the optical system to an ideal one.
Diffractive lenses add degrees of freedom to the design of the system, allowing to improve it in different aspects. With N surfaces, 2N points could be focused. Also less than 2N points could be focused and use the degrees of freedom to correct aberrations, for example, chromatic ones. At this stage of the project, a synthesis method has been developed to obtain the profile of one diffractive surface to focus 2 points. Also a method to correct chromatic aberration using those surfaces has been developed. Fig. 2 shows an example of design that focuses 2 points with a single diffractive surface.
Fig.2. Example of a system with one diffractive surface that focuses two points.
Activity 3: Design of optics for multi-channel architecture.
One of the advantages of the multi-channel architecture is the possibility to develop optical designs equivalent in performance to the conventional ones, but with shorter focal distance. This property is fundamental for Virtual Reality technology, because it allows the design of much more compact headsets and, accordingly, more comfortable for the users.
In the case of Virtual Reality, only designs with reduced number of channels have been used (2 to 4). Reducing the distance display-lens is a challenge for optical design, as shown in Fig. 3. This figure also shows two examples of designs developed in OPTIVAR: one with 2 channels (together with the prototype manufactured) and one with 4 channels (to be manufactured).
Fig.3. Example of optical design for VR with 2 and 4 channels
In this activity, we have designed and developed prototypes for a large set of lenses (more than 30). The objective is the application of those designs to VR systems, to improve some of their parameters. As initial activity, we have designed and manufactured some prototypes for lighting applications, particularly, homogenization of LED’s. Fig. 4 shows one example inspired in insects’ eyes.
Fig.4. Prototype of multichannel system and its application as light homogenizer
Activity 4: Design and development of a Light-Field prototype.
One of the objectives of the project is the generation of Light-Fields using arrays of lenses. The development of this objective will be carried out in several phases, first exploring the market and purchasing displays with the highest resolution possible. Second, developing the Light-Field with the help of a mask of points of different size and position. Finally, the mask will be substituted by an array of lenses specifically designed and with the same performance. The main difference is that the mask is not very luminous, while the array should be as luminous as the original display image. As an intermediate phase, commercial arrays of lenses will be used to start the analysis. The objective of the prototype is to see images in 3D using conventional displays without heasets/glasses. Fig. 5 shows the prototype developed so far.
Fig.5. Array of lenses used for the prototype of Light-Field displays.
Activity 5: Manufacturing of optical components.
In order to manufacture the prototypes of lenses, as the one shown in Fig. 3, different technologies and materials have been analyzed, depending on the necessary quality and cost. The prototypes have been developed using Direct Cut technology by diamond turning. This technology offers results very close to the theoretical design and can be used to verify it. For pre-industrial series, plastic injection molding (with Zeonex material) should be used.
The lenses have been characterized in the optics laboratory at CeDInt-UPM, including mechanical parameters (profile and roughness) and optical parameters (efficiency, aberrations, etc.).
Activity 6: Developement of display control electronics.
We have designed and tested control electronics for two displays for the VR application: one of the operating at 60Hz and the other one, operating at 72Hz. Although a refresh frequency of 60Hz is enough for the response of the eye, it is important to take into account other parameters, in particular, the delay between the moment when the position of the head is obtained and the moment when it is processed. After theoretical analysis of the problem, we concluded that the frequency of the display is very important. Fig. 6 shows the electronics developed working at 72Hz, under testing at this moment.
Fig.6. Testing of control electronics for 72Hz displays. The shape of the electronics is fitted by design to the shape and size of the headset.
Activity 7: Design and development of a prototype headset and integration.
We have developed different designs of the headset, with different materials and colors. Fig. 7 shows two examples, one of them integrating optics, displays and headtracker. The problems of isolation (to avoid dust and light inside the headset), ventilation of circuits and inter-pupillary distance have also been taken into account during the design phase.
Fig.7. Two examples of headset developed to integrate the optics and electronics for VR.
Activity 8: Functional tests of prototypes
During the development of the project, continuous subjective tests are carried out with end users, to receive feedback about comfort, immersivity, quality of image, chromatic defects, tracking accuracy, etc.