Does AR navigation in shopping malls really work?
There might be a clue in the fact 99% of shopping malls don’t offer AR navigation or AR wayfinding in their mobile apps (yet). Or it is because they missed the point of AR in the first place?
The problem: the bigger it gets, the harder to find what you need
The internet economy has made consumers extremely impatient and for a good reason: everything is fast and convenient online. In real life, when it comes to finding a product or place, we want everything to be as easy as looking it up on Amazon or Google maps. But real-life… can be a pain! When going to a mall for the first time, it can get very confusing to find what you’re looking for — even when you only need the washroom! Finding a place or product boils down to 3 basic steps: where am I? Where’s that place/product? How to get there?
Current ways to navigate in shopping malls
1-Ask staff
The good old way works (well) in luxury retail, where you have 3 shop assistants for 1 client. Luxury malls also endeavour to keep a human presence, but outside those premium environments, you’re lucky if there is still one single human soul to assist you. Your best chance is to give it a shot with the cleaning workers or security staff — when they haven’t been replaced by robots 🙃
2- Information kiosks
They usually look fancy with their 3D maps, full of bright colors and animations. Let’s say you want to go to a newly open small little fashion boutique, randomly called RAZA. When clicking on the RAZA icon from the touch screen, a virtual path appears, detailing that you need to go backward for 80m, then go right but only with 45° angle, take the second escalator because the first only goes to the third floor, walk another 65m, turn slightly left and another 50m walk will bring you there. Voila. Good luck finding RAZA 2 min after leaving that info kiosk. The sad truth is they are not useful and turn up to be just another advertising touchpoint.
3- 2D maps
Google/Apple maps work indoor! Not really, despite endless new announcements every year. But to be fair, malls have been trying to create their own 2D maps to provide a solution to their visitors. The variety of technologies used (from beacons to GPS or wifi signal) makes the results very diverse. Overall 2D maps do help, but they fundamentally pose the same issues as any 2D maps, both digital and even paper maps: the human brain has developed a 3D pose scanner (we do that in a few milliseconds by looking around) that is badly designed to pin that spatial understanding against a 2D representation. That’s why so many people still get confused with 2D maps, hence the new feature in Google maps: Live view.
Behind maps: position and interface
The position
Where am I, precisely?
GPS coordinates give a first approach with a +/- 10 meters accuracy (civil application, can’t obtain the military stuff). Good enough to know in which building to enter, even though in highly dense cities like NY, HK or Shanghai that blue dot might jump to +/- 100 meters… But let’s imagine you found the mall you were looking for.
Within the mall, several technologies take over (they are usually working in parallel): BLE beacons and wifi signals mainly, sometimes also earth magnetic field, short-range radio technology like Ultra-wideband (UWB) in particular use-cases. Combining all those technologies should give an accuracy of around 1 meter but with still one big disadvantage: the position will constantly jump, even when the visitor is static — the signal is picked by the receptor (the phone) but constantly juggling between various emitters. On the product level, RFID can be used to track objects, but not users.
Lastly, the blue dot representing a position will still lack crucial information: what is the phone orientation? That orientation is typically given by an arrow and it using various sensors: magnetic and gravity fields + accelerometers. The accuracy of that orientation is a few degrees, considered good enough. So the phone can give us the north, but where is the north in a mall? Do we see big signs THE NORTH IS HERE?
The interface
2D or 3D? We’ve touched upon it, people have a hard time switching from their natural 3D representation of the world to a 3D representation. Figuring out what’s that jumpy blue dot with a little arrow on the top, pointing at a 2D floor representation shouldn’t require an architecture degree. The shift from 2D interfaces to 3D interfaces is more than just an evolution. It’s a fundamental way to rethink about the shopper’s experience starting from a basic human need: to know where we are. Since we position ourselves in our physical environment by looking around, wouldn’t that something a phone could do as well?
Start again with a clean state: human-centric approach
As we saw, the problems 2D maps are trying to fix can be summarized like this: 1- Where am I? 2- What’s around me (that I need/like?) 3- How do I go there?
We’ve insisted on the position and orientation technologies because their limitations are becoming insurmountable in 3D. But we’ve also seen that the phone camera can also replicate our eyes, a technology often called visual positioning system (VPS). VPS replaces every other positioning technology (GPS, beacons, wifi, etc) AND provides an orientation of the phone. This is called the pose and is the starting point to answer the first problem, where am I. Having the pose, developers can now code new 3D interfaces, anchoring information in the physical world. They don’t need to invent 2D representations, symbols or icons, they can directly use a 3D model that looks exactly the same as a real object. Another major advantage is the size of the interface: instead of a small phone screen, they can use the entire physical space, adding content without any physical limitation. As with any interface, it can be personalized to display only what a shopper might want. Brands and marketers now have a way to reach visitors before they even enter a store.
Reveal the unseen. Shopping malls have difficulties driving traffic where they would like, typically to the highest floors. Since visitors can not evaluate the potential gains of going to the 6th floor, they don’t that effort. AR interfaces, when designed with the objective of revealing the unseen, are the only scalable solutions to offer a preview of the real physical experience.
“People seek experiences over products” a well-established maxim but far from a reality today
AR navigation is just an excuse for the real world-scale AR experiences
Creating value by offering new experiences. Eventually everyone finds their way within a mall. AR wayfinding and AR navigation are just an excuse to let visitors open their phone camera and start a new exploration, a journey that blends the physical and virtual. Malls no longer are the transaction place, which increasingly happens on ecommerce. With those long-term online trends, malls are forced to change their entire business model. Can new entertainment revenues cover for the losses in retail sales?
Entertainment can take various forms and certainly Augmented reality wouldn’t be at the top of one’s mind. However, this is changing with the constant improvements pushed by innovators such as Niantic, Nreal or MadGaze just to name a few. World-scale AR experiences are sharing common features: they are context-aware, location-based with few centimeters accuracy, they keep 3D content persistent in space and increasingly can offer real-time multiplayer function. So by definition, playing a world-scale AR experience would require users to visit malls. And hopefully save them from the retail apocalypse!
Augmented reality (and the computer vision, machine learning and gaming technologies behind it) is our core competences, but some of our team members are coming from a retail background, so feel free to drop us a line at contact@neogoma.com if you would like to map a mall and augment it.
