More than four years after I first caught wind of their tech, CREAL’s light-field display continues to be one of the most interesting and promising solutions for bringing light-fields to immersive headsets. At AWE 2023 I got to check out the company’s latest tech and saw first hand what light-fields mean for immersion in AR headsets.
More Than One Way to Focus
So first, a quick recap. A light-field is a fundamentally different way of showing light to your eyes compared to the typical displays used in most headsets today. The key difference is about how your eyes can focus on the virtual scene.
Your eyes have two focus methods. The one most people are familiar with is vergence (also called stereoscopy), where both eyes point at the same object to bring overlapping views of that object into focus. This is also what makes things look ‘3D’ to us.
But each individual eye is also capable of focusing in a different way by bending the lens of the eye to focus on objects at different distances—the same way that a camera with only one lens focuses. This is called accomodation.
Vergence-Accommodation Conflict
Most XR headsets today support vergenge (stereoscopic focus), but not accomodation (single-eye focus). You may have heard this called Vergence-Accomodation Conflict; also known to the industry as ‘VAC’ because it’s a pervasive challenge for immersive displays.
The reason for the ‘conflict’ is that normally the vergence and accommodation of your eyes work in tandem to achieve optimal focus on the thing you want to look at. But in a headset that supports vergence, but not accomodation, your eyes need to break these typically synchronous functions into independent functions.
It might not be something you ‘feel’ but it’s the reason why in a headset it’s hard to focus on things very near to you—especially objects in your hands that you want to inspect up close.
The conflict between vergence and accommodation can be not just uncomfortable for your eyes, but in a surprising way also rob the scene of immersion.
Creal’s Solution
And this is where we get back to Creal, a company that wants to solve the Vergence-Accommodation Conflict with a light-field display. Light-field displays structure light in the same way that we see it in the real world, allowing both of the focus functions of the eyes—vergence and accommodation—to work in tandem as they normally do.
At AWE 2023 this week, I got to check out the company’s latest light-field display tech, and came away with an added sense of immersion that I haven’t felt in any other AR headset to date.
I’ve seen Creal’s static bench-top demos before, which show static floating imagery through the lens to a single eye, demonstrating that you can indeed focus (accommodate) at different depths. But you won’t really see the magic until you see a light-field with both eyes and head-tracking. Which is exactly what I got to do this week at AWE.
On an admittedly bulky proof-of-concept AR headset, I got to see the company’s light-field display in its natural habitat—floating immersively in front of me. What really impressed me was when I held my hand out and a little virtual turtle came floating over to the palm of my hand. Even though it was semi-transparent, and not exceptionally high resolution or accurately colored, it felt… weirdly real.
I’ve seen all kinds of immersive XR experiences over the years, and holding something in your hand sounds like a banal demo at this point. But there was just something about the way this little turtle looked—thanks to the fact that my eyes could focus on it in the same way they would in the real world—that made it feel more real than I’ve ever really felt in other headsets. Like it was really there in my hand.
The trick is that, thanks to the light-field, when I focused my eyes on the turtle in my hand, both the turtle (virtual) and my hand (real) were each in proper focus—something that isn’t possible with conventional displays—making both my hand and the turtle feel more like they were inhabiting the same space right in front of me.
It’s frustratingly impossible to explain exactly how it appeared via text alone; this video from Creal shot through-the-lens gives some idea of what I saw, but can’t quite show how it adds immersion over other AR headsets:
It’s a subtle thing, and such added immersion probably only meaningful impacts objects within arms reach or closer—but then again, that distance is where things have the potential to feel most real to use because they’re in our carefully watched personal space.
Digital Prescriptions
Beyond just adding a new layer of visual immersion, light-field displays stand to solve another key problem, which is vision correction. Most XR headsets today do not support any kind of prescription vision correction, which for maybe even more than half of the population means they either need to wear their correctives while using these devices, buy some kind of clip-on lens, or just suffer through a blurry image.
But the nature of light-fields means you can apply a ‘digital prescription’ to the virtual content that exactly matches the user’s corrective prescription. And because it’s digital, this can be done on-the-fly, meaning the same headset could have its digital corrective vision setting change from one user to the next. Doing so means the focus of virtual image can match the real world image for those with and without glasses.
Continue on Page 2: A More Acceptable Form-factor »
,
A More Acceptable Form-factor
While the head-worn prototype I wore was compelling, it wasn’t the newest thing that Creal is showing. That headset provided a proof-of-concept for the light-field itself, but clearly used bulky hardware and optics that wouldn’t be suitable for fitting into a practical headset.
To that end the company is now showing off that it can make its light-field display work with any standard lens thanks to a holographic coating which acts as the ‘lens’ to get the virtual image into your eye.
This approach enables much more transparent lenses and fits a much more common glasses-sized form-factor—both of which are key for social acceptance which is a big deal for any head-worn device that is used outside the home. And further, Creal says the holographic coating can be applied to existing ophthalmic lenses, meaning a standard corrective glasses lens can form the basis of the display.
That’s a serious advantage considering that traditional ophthalmic lenses are already manufactured at scale, and are a known quantity in terms of cost, durability, social acceptability, etc.
Challenges Remain
But Creal’s light-field display still has some drawbacks compared to the conventional displays seen in other headsets today.
For one, the resolution isn’t as good as what’s been achieved with displays in other headsets. Creal says its tech can achieve higher resolution, but what it’s managed to demo so far isn’t as sharp as contemporary headsets on the market today, let alone next-gen devices poised to launch. Further, the system struggles to achieve an acceptably wide color gamut, giving images a somewhat washed-out look.
Granted, the company has been continuously improving nearly every aspect of the system’s performance over the years, and it believes there’s still significant improvements to be made as it refines its methods, components, and calibrations.
Creal has been transparent about its performance goals; here’s where the company claims it is today and where it thinks its tech can get to by 2025:
| (per eye) | 2023 | 2025+ |
| Angular resolution at infinity | 40 PPD | 50 ppd |
| Modulator resolution | 1 Mpix | 1+ Mpix |
| Depth resolution | Continuous | Continuous |
| FoV (diagonal) | 36° | 60° (light-field: 20-30°) |
| Effective eyebox (exit pupil) | 13 mm (6 mm) |
16 mm (10 mm) |
| Eye relief | 20 ± 3 mm | 20 mm |
| Colors | 2 M | ~10 million |
| Rendering load (equivalent to flat image) |
HD | n/a (on chip) |
| Frame rate | 160 Hz | Up to 240 Hz |
| Sub-frame rate | 6.5 kHz | 8 kHz |
| Brightness | 2000 nits | Up to 7000 nits |
| Contrast | 1 000/1 | 10 000/1 |
| Combiner type | Holographic, prescription compatible | Holographic, prescription compatible |
| Transparency | 91% | 97% |
| Power consumption | 800 mW | ~100 mW |
| Volume module (LF. engine) | 7 cm3 | 2 cm3 |
Field-of-view, however, is perhaps the system’s biggest challenge. As you can see in the specs above, although Creal expects to achieve a 60° (per-eye) FoV, only 20-30° will be a genuine light-field. The rest can be filled out with non-light-field imagery to expand the overall view, but the outer portion won’t have the same light-field benefits as the center part.
However, our eyes are rarely pointed directly ahead. Instead they regularly swivel off-axis to look at things during regular day-to-day life (and when glancing around a virtual scene). Without having the light-field cover a more significant portion of the field-of-view, this defeats some of the benefit of the display.
Creal has said it hopes to overcome this issue with ‘pupil steering’ (also known as dynamic foveated display), which would track the user’s eye and move the light-field portion of the view to make sure it’s always at the center of your gaze. However, this is another significant technical challenge for the industry at large, and one that I’ve yet to see sufficiently solved in a form-factor that would work in a glasses-sized product.
Still, it’s exciting to see a company working hard on this challenge that others seem to be leaving untouched, so we’ll continue to keep our eyes on Creal.