Microsoft HoloLens 2 offers trippy, eyeball-tracking augmented reality
On Tuesday, Ars had the opportunity to sit down with Microsoft's Director of Communications, Greg Sullivan, to walk through a quick demonstration of the company's new mixed reality headset, the HoloLens 2. HoloLens 2 isn't a full-immersion virtual reality device like Oculus Quest; it's an augmented reality device—think Pokémon Go on steroids. Sullivan never used the word "augmented" at all, preferring instead to talk about "mixed reality."
When I asked the difference, he explained that the majority of the content and techniques a user experiences with HoloLens2 would translate directly to a hypothetical full-immersion VR headset with the same sensors. "Mixed reality" is a blanket term that encompasses both augmented reality—which specifically mixes elements of virtual and real world—and virtual reality, which at least in theory immerses you entirely in the virtual. Sullivan went on to point out that even what we usually think of as fully virtual reality can't avoid the real world entirely—the six degrees of freedom (6DOF) a VR user experiences end abruptly if the user bonks into a real-world wall.
Ergonomics and eye tracking
If you're accustomed to VR headsets, one of the first striking things about HoloLens 2 is its weight—or lack thereof. Most VR headsets feel something like a diving helmet—they're heavy and oddly balanced, turning every VR session into a neck workout. HoloLens 2 is considerably lighter, easier to adjust, and pretty much perfectly balanced. It has a single adjustment knob in the back, like a hard-hat or welder's mask, and it slides on and off easily. Adjusted properly, it "floats" across the bridge of the nose, resting a little but not all of its weight there.
With the device on and adjusted, Sullivan led me through its features. I paid particular attention to tracking—most mixed reality devices know where you're pointing your head but have no idea what you're actually looking at. HoloLens 2 actually tracks your eyeballs themselves, so it knows what you're really looking at, and even how far away it is. The effect is uncanny—in one demonstration I was asked to read a block of text, which automatically scrolled down as my eyes tracked across it.
The result of this eyeball-tracking reading experiment felt oddly like telepathy—slightly frustrating telepathy, in this case. The text scrolls automatically as you look directly at the bottom row of text in a block of six or so rows; I normally read two or three rows in a glance, so it felt like needing to look slightly past the bottom to make it scroll.
Obviously, this is something that can be easily tuned—but it's intriguing how unfair it feels when something seems like a psychic power but still doesn't quite understand you.
The lack of handheld controllers with HoloLens 2 also grabbed my attention—I'm used to pinching and grabbing, but with most VR rigs, this is done with the aid of handheld controllers that track your hands' position and give you triggers to squeeze. That effect eventually feels pretty natural, but it's unnecessary with HoloLens 2, which instead tracks your bare hands' position and actions visually. You can reach out and pinch hologram elements between thumb and forefinger, tap buttons with an extended finger, or "visually tap" far away items by looking at them, then pinch your thumb and forefinger outside your own field of vision to "click."
HoloLens also gives you the chance to feel like a cyborg; several parts of the demonstration utilize a virtual control panel mounted on your left wrist. If you raise your hand with palm facing you and look at your left wrist, you get a series of buttons apparently embedded in your arm which can be tapped; some applications also offer a larger, hand "held" control panel which appears inside your opened left hand.
These virtual control panels came in handy in a simple drawing app, which allows a user to free-form sketch three dimensional shapes in the air. Depending on the tool selected from a palette in your left hand, you can draw, rotate, clone, or move these free-drawn shapes; I very quickly littered the whole room with bizarre arcs and patterns only I could see.