Making Smartphones See

How Google’s Tango Augmented Reality platform is changing the way we interact with the world.

- James Lu & Marcus Wong

For the past six years, Google’s Advanced Technology and Projects (ATAP) team has been hard at work on an augmented reality computing platform that they say, “could one day be as essential to your phone as GPS”. That platform is Tango and it’s finally here. And it’s about to change the way you and your smartphone interact with the world.

Google’s version of Augmented Reality is a huge leap forward from Pokémon Go or any other AR app you may already be familiar with. Announced early in 2014, Tango is Google’s attempt to get mobile phones and tablets to see the way the human eye sees. This is no simple task. It involves an extensive camera array that uses computer vision, depth sensing and motion tracking sensors to grant the device full spatial awareness; in other words: the ability to understand your environment and your relation to it. Tango doesn’t require GPS or any other external signal, which means it can do indoor navigation – something that’s never been done on a mobile device before. Current VR devices like the HTC Vive require carefully calibrated external sensors to know where you are, but with Tango’s ‘inside out tracking’, everything you need is inside your smartphone. And if that’s not impressive enough, Tango doesn’t just know where you are, it also maps and tracks every single 3D object in the same room as you.

Google’s first Tango device was a large, bulky prototype tablet made available only to developers back in 2014. Since then, Google has formed a partnership with Lenovo to bring a Tango-enabled device to consumers, packing all of the sensors and processing required to run Tango into a portable, consumer-friendly form factor. That device is finally here: the 6.4-inch Lenovo Phab 2 Pro phablet.

How does it work?