Apple Vision Pro Does Everything Well but Immersive Gaming

While Apple’s new Vision Pro headset isn’t going to satisfy the existing base of consumer VR users, it’s mastering the rest of the basics better than anyone else.

Probably 90% of what consumers are using VR headsets for today is entertainment, and of that entertainment, most of it is gaming. And if you’re among those people using such headsets today, you’ll reasonably be disappointed that Apple Vision Pro lacks controllers and isn’t going to be playing top VR games anytime soon. But for everyone else, it’s a back-to-basics approach that’s laying a sturdy foundation to build upon in the future.

Today at Apple’s headquarters I got to check out Vision Pro for myself. Unfortunately the company didn’t permit any photos or footage during the demo, but the clips below are a fair representation of what I saw.

Photo by Road to VR

Apple Vision Pro (APV, let’s call it) is doing what only Apple can: carving out a subset of what other devices do, and making sure that subset of things is done really well. And given the current state of UX on most other headsets, this is a reckoning that was a long time coming.

Look & Tap

It starts with the input. Apple is leaning heavily into using your eyes as a cursor, and a pinch gesture as a click. The headset has cameras on the bottom that face downward so that even subtle pinches from your hand in your lap are visible and detected. But you don’t see a floating cursor where your eyes are, nor do you see a laser pointer shooting out of your hand. You just look at the thing you want to press, then do a quick pinch.

‘Hello Neighbor VR’ Coming to All Major Headsets Soon, Gameplay Trailer Here

On paper you might think this sounds shoddy. But remember, this is Apple. They’ve tested and refined this system six ways from Sunday, and it works so well that after a minute or two you hardly think about how you’re interacting with the headset, you just are.

The pinch input is responsive and reliable. It felt so natural that the two or three times the headset missed my pinch during a 30 minute demo it felt really weird because my brain was already convinced of its reliability.

This look-and-pinch system is so simple for the headset’s basic input that I won’t be surprised if we see other companies adopt it as soon as possible.

Reality First

So there’s the simple input and then there’s a passthrough-by-default view. This is an MR headset after all, meaning it can easily do augmented reality—where most of your view is of the real world, with some virtual content; or virtual reality—where all of your view is virtual content.

When you put AVP on your head, you instantly see the outside world first. In fact, the way that Apple defers to the passthrough view shows that they want to treat fully immersive experiences as the exception rather than the rule. Generally you won’t pop into a fully immersive scene unless you actively making the decision to do so.

The passthrough view is certainly best-in-class, but we’re still probably two generations away from it truly feeling like there’s nothing separating your eyes from the real world. Granted, I was able to read all the text on my phone with no issue, which has been the ‘bar’ for passthrough quality that I’ve been waiting to see exceeded.

VR’s Favorite Rhythm Shooter is Getting a Modding Tool Next Month, Open Beta Now Live

Beautiful Virtual Displays

The imperfect passthrough resolution somewhat betrays the exceptional display resolution which exhibits not even a hint of screen-door effect. It may not be ‘retina resolution’ (generally agreed to be around 60 pixels per-degree), but it’s good enough that I won’t know how far off it is from retina resolution until I sit down with an objective test target to find out.

That’s a long way of saying that the headset’s display has excellent resolution with great clarity across the lens. Top of the class.

This clarity is helped by the fact that Apple has done its Apple-y thing and ensured that panels, text, and images consistently render with superb quality. The entire interface feels iOS-polished with animations and easy to use buttons and controls. The interface was so simple to use that the demo chaperones had a hard time keeping me on task as I wanted to flick through menus and move floating apps around the room.

But here’s the thing, probably 75% of what Apple showed me was essentially just floating screens. Whether it was videos or a floating iMessage app or the web browser, it’s clear that Apple wants Vision Pro to be first and foremost be great at displaying flat content to the user.

The other 25% of what I saw, while very impressive all around, felt like just the start of a journey for Apple to build out a broader library immersive experiences.

Record & Rewatch Memories

AVP might not be a VR gaming headset, but it does at least one thing that no other headset does: capture volumetric memories using its on-board cameras. Using the button on the top of the headset you can capture volumetric photos and videos with just a press.

Meta Keeps the Oculus Name Alive as Third-party VR Publisher Becomes ‘Oculus Publishing’

Apple showed me a demo of a volumetric video capture of a group of kids blowing out candles on a birthday cake. It was like they were right in front of me. I’d never even seen these kids before but I could immediately feel their giddy emotions as they giggled and bounced around… as if I was sitting right there while it was happening. Not to mention that the quality was good enough, at least in this best-case-scenario demo capture, that my first thought had nothing to do with the famerate or quality or dynamic range, but purely of the emotion of the people in front of me.

That instant connection—to people I don’t even know—was a clear indicator that there’s something special to this. I can already imagine watching a volumetric video of a cherished memory, or of a loved one that has passed, and I know it would be a powerful experience.

Doing it Right

And here’s the thing; I’ve seen plenty of volumetric video demos before. This isn’t a new idea, not even close. The thing that’s novel here is that everyday users could potentially shoot these videos on their own, and readily watch, share, and store them for later. On other headsets you’d need a special camera for capturing, special software for editing, a player app, and a sharing app to make the same thing happen.

This is the ‘ecosystem’ part of XR that’s missing from most other headsets. It’s not about what’s possible—it’s about what’s easy. And Apple is focused on making using this headset easy.

Continue on Page 2: Immersion Isn’t Off the Table »


Immersion Isn’t Off the Table

While AVP clearly isn’t being positioned as a gaming machine, Apple did show that it’s interested in fully-immersive content, even if it doesn’t expect that to be the most common usage right out of the gate.

In addition to allowing you to swivel the ‘digital crown’ dial on top of the headset to fade into a fully immersive environment as your background, the company also showed an impressive demo where the wall in front of me opened up to reveal a prehistoric environment where a large dinosaur came up to the opening and stuck its head into the room.

The dino, which was rendered in real-time with impressive detail, dynamically reached its head toward me to sniff and snort while investigating me up close. If I moved, it moved to follow me. Just before it looked like it was going to step all the way into the room, it turned around and sauntered off.

For people who have never seen VR before, this will probably be a mind-blowing encounter. As someone who has been immersed in the tech for the last decade, the demo wasn’t anything I hadn’t seen before, but as with the rest of what I saw, the execution was spot-on and the effect was very convincing. Thanks to a great display, convincing passthrough, seamless tracking, and sharp 3D rendering, this was one of those moments where I was more focused on the feeling of Presence—the feeling that this thing was really there in front of me—than the underlying tech at work.

Content Consumption

While Apple has steered away from particularly interactive immersive experiences thus far, the company still wants to entertain. To that end they showed a demo reel of Apple Immersive Videos, a collection of 8K 180-degree 3D live-action videos.

This was a quick-cut reel of a bunch of scenes: a bear in the wild, children playing soccer, a basketball game, an exotic city, a daredevil on a tightrope between two mountains. It was a great showcase of the power of immersive video. Again, I’ve seen this sort of stuff plenty of times over the years, but between the quality of the capture and the fact that this content library is going to be built into Apple TV, there’s a real possibility that this kind of content actually gets viewed, shared, with ease. Unlike most other approaches we’ve seen to date which have relied on third-party apps or poorly constructed content libraries that nobody knows how to find.

If I found an awesome immersive video on AVP and wanted to send it to a friend, I could just click ‘share’ and send them a link via iMessage right from within the headset. Or maybe I’ll just be able to invite them to simply watch it with me. That’s the difference between what Apple is building with AVP and what’s come before. It’s supporting known paradigms with the goal of making obvious things (ie: watch an immersive video with your friend) not just possible, but easy.

The Buts

But… there’s a few buts.

For one, I was hoping Apple would move the needle more on headset ergonomics. AVP isn’t uncomfortable compared to contemporary MR headsets, but it’s still in the same ballpark. I found the headset fairly front-heavy, even with an optional top strap. I’m glad they have the top strap though, because AVP (like most headsets) definitely benefits from it. However, the choice to have the strap go from ear-to-ear (rather than forehead to back-of-head) seems like a mistake because it doesn’t let you lift the headset’s front-heavy weight up quite as well.

Image courtesy Apple

It’s difficult to judge headset ergonomics without real long-term usage, but my gut was that Quest Pro is more comfortable thanks to its more balanced design and rigid headstrap.

Luckily AVP’s headstrap is designed with a simple release mechanism that should make third-party headstraps plentiful. And I wouldn’t be surprised if the kind of headstrap people gravitate toward the most includes a top strap that goes from forehead to back-of-head, as well as a mount on the back of the head for the battery (to better balance the weight).

Photo by Road to VR

And let’s just get this out there: it’s a big bummer that AVP lacks dedicated motion controllers—even as an optional add-on. The most popular VR games all require VR controllers (and will be painful, if impossible to port for hand-tracking alone). The hardware overall is generally pretty awesome and you can be sure I’d love to play my favorite VR games with it. Sadly that’s not possible and it looks like it won’t be any time soon.

The Price is Insane. Period.

$3,500 is insane for 99% of people, and Apple knows it.

The Pro in Apple Vision Pro indicates as much… this device is for ultra Apple enthusiasts, exceptionally wealthy people who want the latest and coolest tech money can buy, and developers who think they might be able to build a business from the headset and its eventual progeny.

Report: Apple’s MR Headset Could Come as Soon as This Summer

With this first-gen headset, Apple has planted a flag in the ground with Vision Pro, saying, ‘we want the MR experience to be at least this good, regardless of what it costs’. From here the goal is clear: make it smaller, better, and cheaper. And if they keep doing that, future iterations of AVP will become increasingly valuable and increasingly accessible to more and more people. If this is the bar they’re setting for themselves (and by extension, the rest of the XR industry), then the future is looking very bright… even if it’s not quite here yet.