A recipe for success with Apple VR headset

Almost every major player in technology has been dreaming of smart glasses in recent years. Of Amazon’s Echo Frames to Facebook’s plans for augmented reality, to Microsoft HoloLens and the ongoing reports of Apple’s AR plans – there seems to be a lot of boiling. But in the short term, the road to the magical Tony Stark glasses will be more likely through VR. This is exactly what Mark Gurman of Bloomberg reports as the next step from Apple: a pair of advanced VR glasses it can be popular in some ways like Facebook Oculus Quest.

Apple has never before had a VR headset (or glasses), and lies large over the landscape with a growing interest in augmented reality. Apple also bought the VR company NextVR last May. Google, meanwhile, has been making its own VR headset for years, even acquiring VR software companies behind apps like Tilt Brush and Job Simulator. But Google has since moved away from VR and focused on AR.

VR hardware is already here and it’s surprisingly good. The Oculus Quest 2 is one of my favorite game consoles and is easy to set up and dive into. But VR everywhere else is still experiencing growth pains. PC VR has great games and some excellent headsets, but it’s still awkward, wired, and requires a lot of specific hardware to work (and still feels viewed on Windows, instead of organic). The new PlayStation 5 currently treats PlayStation VR, originally released in 2016, as an afterthought.

What is needed at the moment is not just better working technology: it is deep support in apps and phones, so these headsets can feel like they are talking to everyone else in your setup.

Most VR headsets feel stuck on their own islands: specific games, apps, and hardly any cross-communication between computers or phones, or anything else. They are shockingly removed from the ecosystem of mobile devices – smartwatches feel more trapped in my life than any VR headset has ever done. For many arts and experiences that VR does best, this disconnect works well. It’s like an isolation tank carried by the head. But it’s very bad to make VR any kind of tool you’ll ever want to use in everyday life.

VRs have always felt like a headphone to my eyes, a way to extend a view around me and dive more fully into something. Headphones are plug-and-play and work with just about anything you need. VR headsets do not. But maybe they could. Maybe this is exactly what Apple – and even Google – needs to activate next.

AirPods Max for your eyes: a fantastic headdress

Apple’s recent headphones are already moving a clear path. The AirPods Max, with their spatial sound and extremely high price, go for design, immersion and fidelity. The same will be the case with any VR headset. VR screens are gradually getting better and are now bright enough to see almost no pixels. But it still is no better than the phone screens and monitors and TVs we use all the time. There’s only one VR headphone I’ve ever seen, by Varjo, that looks ‘retina level’ enough to be just as good or better than a giant monitor. If we know Apple, it would seem like it’s a big goal to go for ‘the best display’, or to promise it.

VR headsets must work with iOS (and Android) phones

Yes, phones have already had a VR glasses phase. But the mid-2010s accessories were basically plastic lenses that you attached to your phone and were very basic compared to what devices like the Oculus Quest can do.

But now, VR headsets like the Quest barely talk to phones and rather manage their own apps. Facebook’s walled garden of content, which requires a Facebook account, should work better with other devices if it’s going to be a better home office tool, something Facebook is planning. Facebook can get there, but collaboration from Apple (iOS, Mac) and Google (Android, Chrome) is needed. Even Facebook’s relationship with Windows, when it comes to managing Oculus apps, feels divided.

Phones and tablets are already super powerful and can work as complete computers with VR headsets as peripherals. Apple can expand iOS into headsets and move to the iPhone, iPad and even the Watch to combine the experiences.

Google started exploring VR with its Daydream platform and then stepped back. Even if Google does not make its own VR ecosystem, Android should play better with headsets that want to communicate.

Apple’s chips can work on larger VR headsets first and later invent smaller glasses

Qualcomm’s recent moves over the past few years, which have seen the light of day for lightweight headphones and glasses to connect with phones and eventually stream graphics over 5G, look like what the future may bring. Qualcomm’s processors are currently in just about every VR and AR headset: the Oculus Quest 2, Microsoft HoloLens 2, Vuzix’s next pair of glasses.

Apple’s clear path involves developing or customizing its own extremely powerful phone and Mac chips to shape the headphones and, in a sense, do what Qualcomm did. Or something that is possibly more advanced.

But smart glasses that AR do are still years away, according to almost every company I spoke to. A VR headset with AR effects that blends reality with cameras that can see the outside world is a clear middle ground.

Lidar is already here: it can push VR headsets and also mix reality

Apple has already placed lid sensors on its pro iPads and iPhones, using depth detection to map out a room and ‘switch on’ to create a 3D map of what is there, so that virtual objects can be layered word. Headsets like the Oculus Quest 2 can search room barriers to some extent with basic external cameras, but lidar can do this much more accurately and possibly faster.

An upcoming pro VR headset manufactured by the Finnish company Varjo has a lidar, which also makes it possible to scan the world and bring real objects into VR.

What I really want is a tracking system that can prevent me from bumping my hands into a wall or desk, but that the lidar needs to be constantly on the go. I’m not sure if Apple’s lidar can do that without a great battery life.

Would Apple invent a better VR controller?

The Oculus Quest 2 has excellent controllers, but they feel more like a gamepad game console – buttons, analog sticks, triggers. It’s good for games, not good for making a VR headset in a next generation computer. Would Apple use hand tracking and air gestures, as the Quest and Microsoft HoloLens already have, or develop some kind of worn strap (like an Apple Watch) that could help?

This is one of my biggest question marks for an Apple VR headset, because I can not imagine what they can do.

Something that can seamlessly switch between standalone, phone, iPad or Mac

Just like the AirPods or Apple’s continuity for AirDrop and file and link sharing, a VR headset should work on anything nearby. The Oculus Quest comes closest to this by working alone or with a Windows computer if connected with a USB-C cable.

The more Apple and Google headsets work with their devices, the better off everything will be

Even more than an Apple VR / AR headset, I want the following versions of iOS and Android (and MacOS, and even ChromeOS) to make it possible to connect these things. Without that glue, VR headsets are meant to always feel like strange toys. I’ve been using VR a lot more than I ever expected. But if I’m ever going to use it for anything more, or if Apple intends to place its hardware as a creative tool, it should work well with everything else.