The sub-$300 Quest 2 is a spectacular virtual reality (VR) headset, allowing budget-conscious consumers to access the VR world without spending a fortune. Does it perfectly mimic reality? No, but to be fair, there isn’t a single consumer headset on the market that can. And Meta explained why that is.
At a virtual livestream addressing the press, Mark Zuckerberg discussed that Meta will one day launch the quintessential headset with lightweight comfort and ultra-high fidelity à la “Ready Player One.” However, the Meta CEO admitted that this is easier said than done.
Why current-gen VR headsets are still clunky
Wouldn’t it be awesome to walk through a VR game environment, like a horror title, that seems so realistic, your brain can’t perceive what’s real and what’s not? How cool would be to see a photorealistic hologram of your favorite singer dancing in your living room via augmented reality (AR) tech?
These are some of the grandiose goals Meta has in store for its VR arm, but as mentioned, there are some dilemmas it must tackle first. “Just seeing a realistic-looking image isn’t enough to feel like you’re really there,” Zuckerberg said, discussing the limitations of current-gen headsets. The Meta CEO then listed all the visual cues users need to reach peak realism in the VR world:
- A display that covers a wide angles for a vast field-of-view
- Retinal-level resolution and an ultra-high pixel count
- Brightness levels and dynamic range that mimics the physical world
- Realistic, low-latency motion tracking
- High-performance CPU and graphics
On top of all that, Meta needs to somehow pack all of those powerful internals into a lightweight, comfortable headset, which is difficult and damn-near impossible due to high heat generation. Hence why current-gen VR headsets tend to be clunky. It’s either build an unwieldy head-mounted display that’s cool as a cucumber or a create featherweight one that could burn users’ face off.
Checking all these boxes is making Meta’s head spin, but according to Zuckerberg, the company has a “long-term roadmap that will solve these different challenges.” And Meta intends to do so by introducing a benchmark called the “visual Turing test.”
What is the visual Turing test?
For the uninitiated, a Turing test is an analysis that determines whether a machine can imitate human intelligence to a point that it can deceive real humans into believing that it’s a living, breathing person.
As such, when Meta uses the term “visual Turing test,” it implies that the social-media giant hopes to one day launch a headset that renders visuals that are so realistic that users can’t figure out whether they’re looking at a simulated environment or the real world.
“It’s a test that no VR technology can pass today,” Meta Chief Scientist of Reality Labs Michael Abrash said. “VR already creates a strong presence in being in virtual places in a genuinely convincing way, [but] it’s not yet at the level where anyone would wonder what they’re looking at is real or virtual.”
One of the biggest challenges that lie ahead for Meta is tackling resolution. “We estimate that getting to 20/20 vision [in VR] across the full human field of view would take more than 8K resolution,” Zuckerberg said.
Fortunately for Meta’s VR engineers, our eyes don’t perceive everything in high resolution across the entire field of view. Consequently, the Reality Labs team will take advantage of this idiosyncrasy. Still, Meta still has the challenge of achieving “retinal resolution” in a headset, which means getting up to about 60 pixels per degree. “This is about three times where are today,” Zuckerberg added.
Meta’s display research team got to work on developing prototypes that could accurately depict the real world. The most advanced model it developed to rectify VR’s resolution issue is called “Butterscotch.” It’s resolution is so high, users can read the 20/20 vision line on an eye chart in VR.
Butterscotch can serve up about 55 pixels per degree, which is 2.5x the resolution of the Quest 2. But is it ready for market? No way. It’s field-of-view is much narrower than standard VR headsets. On top of that, it’s too bulky and heavy.
“Butterscotch” perfectly demonstrates Meta’s dilemma. Sure, it can produce a headset with ultra-high fidelity, but unfortunately, the engineers are stumped on how to shrink it, moderate thermals, make it light enough for long-term wear, and more.
Despite these obstacles, Zuckerberg is hopeful. “We expect display panel technology is going to keep improving. And in the next few years, we’ll have a good shot at getting there.”
Another hurdle Meta is working on to pass the visual Turing test is depth of focus. “To address this, we came up with a way to change the focal depth to match where you’re looking by moving the lenses around dynamically — kind of like how autofocus works on cameras,” Zuckerberg said. “This is known as varifocal technology.”
To put it succinctly, varifocal technology delivers proper depth of focus by relying on eye tracking (so the system “knows” what you’re looking at) and distortion correction (compensates for the magnification that occurs with lens movement). Varifocal ensures that only the virtual artifacts that you’re looking at are in focus — just like the physical world.
Like it did with its experimentation of retinal resolution, Meta created a line of prototypes to research depth of focus and varifocal technology. It’s called “Half Dome.”
Similar to Butterscotch, Half Dome and its varifocal magic is not ready for production; Meta insinuated that its eye tracking is still unreliable.
Other challenges Meta intends to tackle are distortion produced by VR optics and high dynamic range (the overall brightness and contrast of the display). Delving into the latter, Meta explained that the preferred number for peak brightness on a TV is 10,000 nits. However, the Quest 2 can only reach 100 nits. “Getting beyond that in a form factor that is wearable is a challenge,” Abrash said.
One prototype that Meta built that’s likely closest to its “Ready Player One” vision is called Holocake 2, but in some ways, it’s still quite far from reaching the company’s lofty ambitions. The headset requires specialized lasers to run it smoothly and efficiently. “Lasers aren’t super exotic nowadays, [but] they’re not really found in a lot of consumer products for the performance, size and price we need,” Abrash said. Meta is working on discovering a “consumer-viable laser” for Holocake 2 that meets its standards.
Ideally, Meta wants to release a ski goggles-esque headset that includes all the aforementioned technologies it’s been incubating in its lab for the past seven years.
Why does all of this matter to Meta? Well, as you can see with products such as WhatsApp, Facebook and Instagram, Meta’s focus is connecting people on a virtual plane. As such, Meta wants its hardware tech to meet its lofty visions of photorealistic avatars, high-fidelity environments, and more.
If Meta can manage to hop over these hurdles and achieve jaw-dropping realism with a next-gen headset, it’d be a game changer for the Quest 2 maker.
