Virtual, Augmented and Mixed Reality

You’ve probably heard the most about virtual reality, or VR. It’s the technology that has seen big consumer releases within the last few years, in devices like the Oculus Rift, GearVR and many others. People also tend to know VR better because of the attempts to market the technology back in the 90’s when we just didn’t have the technology to make it effective yet.

In VR, you wear something on your head — currently, a “head-mounted display” or HMD that can look like a boxy set of goggles or a space helmet — that holds a screen in front of your eyes, which in turn is powered by a computer, gaming console or mobile phone. Thanks to specialized software and sensors, the experience becomes your reality, filling your vision; at the high end, this is often accompanied by 3-D audio that feels like a personal surround-sound system on your head, or controllers that let you reach out and interact with this artificial world in an intuitive way.

What distinguishes VR from adjacent technologies is the level of immersion it promises. When VR users look around — or, in more advanced setups, walk around — their view of that world adjusts the same way it would if they were looking or moving in real reality.

The main focus here is technology and content that can fool the brain into thinking it is somewhere it’s not. When you flinch at a virtual dinosaur, or don’t want to step off an imaginary ledge, that’s the effect you’re looking for.

Augmented reality, or AR, is similar to VR in that it is either delivered through a sensor-packed device that gives you a window with which to view both your actual surroundings and also the augmentations to your surroundings. Since it was first announced at WWDC, Apple’s ARKit has been open to developers to get a feel for the new platform. Now that iOS 11 is finally here, ARKit is available for anyone with an iPhone 6S or later, the iPad Pro, and the latest 9.7-inch iPad.

The key term for AR is utility. A typical augmented-reality experience will probably be a lot less exciting than meeting a dinosaur or riding a roller coaster, but analysts have argued that the potential market for AR applications is actually much larger than VR’s.

The whole point of that ugly word, augmented, is that AR takes your view of the real world and adds digital information and/or data on top of it. This might be as simple as numbers or text notifications, or as complex as a simulated TV screen. AR lets you see both synthetic objects as well as objects in the real world simultaneously.

AR makes it possible to get that sort of digital information without checking another device, leaving both of the user’s hands free for other tasks. You may see this technology in use today in the form of a heads-up display projecting your speed and direction onto the windshield of your car as you drive.

An extension of augmented reality is what is known as mixed reality. It tries to combine the best aspects of both VR and AR, wrapped up in a marketable term that sounds marginally less geeky than its cousins.

In theory, mixed reality lets the user see the real world (like AR) while also seeing believable, virtual objects (like VR). And then it anchors those virtual objects to a point in real space, making it possible to treat them as “real,” at least from the perspective of the person who can see the MR experience.

With mixed reality, the illusion is harder to break. As you move, the virtual objects in your display may react to your surroundings by keeping position in relation to a real-world object or react in a realistic way to changes in lighting. With this sort of interaction you could see how a new living room set would look in your house or virtually try on clothes while looking into a MR mirror.

It’s only a matter of time before we’re all wearing MR-enabled contact lenses and the line between virtual space and reality is blurred forever.