Reverse passthrough VR: eyes on the outside of your VR glasses.
Do you remember those old holographic glasses that looked like eyes were staring out? You’d wear them and look insane. It’s hard to look at Facebook Reality Labs’ newest experimental project, being presented as part of this year’s SIGGRAPH conference, in any other way.
There’s a reason for the weirdness, but I’m not sure I agree with it.
The concept is called “reverse passthrough VR,” developed by researcher Nathan Matsuda. The experimental headset puts a 3D light field display on the outer part of the headgear, effectively showing glasses-free 3D of what your eyes might look like. It’s almost like putting an iPad in front of your face with a picture of your face looking out. But in 3D.
According to Facebook, the goal is to solve “how we may be able to maintain “social co-presence” in shared spaces between people who may or may not be involved in the same AR/VR experience.” Facebook is basically trying to solve the strange disconnected look of being in a VR headset. When I have an Oculus Quest 2 on my face, I look blindered to the rest of the people around me in my house.
What’s weird, though, is that I’m not always cut off: Passthrough cameras on the Quest 2 let me see out, but no one else knows I’m looking.
The concept looks very off-putting, and something like a novelty. Even Facebook Reality Labs Chief Scientist Michael Abrash confessed skepticism in Facebook’s blog post: “My first reaction was that it was kind of a goofy idea, a novelty at best, but I don’t tell researchers what to do, because you don’t get innovation without freedom to try new things, and that’s a good thing, because now it’s clearly a unique idea with genuine promise.”
An earlier version of the reverse passthrough tech: Matsuda displaying a recreation of his eyes. Looks like a strange Halloween mask.
What is interesting here is that the tech being tested feels like a prototype for future VR headsets, in many ways. The thinned-down display concept, based on Facebook’s still-in-development Half Dome 2 vision for VR glasses, fits eye tracking in and can use that movement to make the projected light field 3D images of someone’s eyes and face move to reflect what you’re looking at, or expressing. Facebook’s working on extremely realistic avatar stand-ins for future telepresence and chat, which is what the rendered eye and face graphics are eventually aspiring to work with, or dovetail with.
As Facebook works its way toward headsets that use passthrough cameras to blend the virtual and real on your face, it’ll help to let others know you’re aware of what’s going on in the outside world (maybe). Eyes, or the illusion of eye contact, could maybe make things less weird. Or would it?
Right now, the experiment looks completely off-putting and unsettling. I’d prefer just never seeing someone’s eyes to seeing a weird representation of their eyes. Of course, I haven’t seen this tech in person. Maybe it feels less uncanny that way.
Facebook is aiming to build better avatars for VR, eventually aiming to make them extremely realistic as possible stand-ins for our actual faces. Chatting in VR, unlike Zoom, can’t currently capture our actual facial expressions with a headset on. This is one of VR’s biggest problems when it comes to dreaming of meetings and social experiences that feel normal: You can’t see your face, so you become a cartoon avatar instead.
Will projecting eyes onto headsets help get past some of the weirdness at home? I doubt it. But Facebook may also be looking to solve how to blend elements of computer-generated avatars with our real selves and see how that works. On our face, in future chat apps or who knows where. Maybe this blend of artificial eyes and real face is part of that process. Right now, I’m still feeling way too much uncanny to accept it.
