Mobile Technology

IBM brings augmented reality to the shopping aisle

View 2 Images
IBM’s Fiona Doherty demonstrates an augmented reality shopping app that automatically delivers personalized coupons, offers, customer reviews and hidden product details (Photo: Jon Simon/Feature Photo Service for IBM)
IBM’s Fiona Doherty demonstrates an augmented reality shopping app that automatically delivers personalized coupons, offers, customer reviews and hidden product details (Photo: Jon Simon/Feature Photo Service for IBM)
Infographic from IBM highlighting the benefits of the augmented reality shopping app currently under development

While the rise and rise of online shopping has eaten into the profit margins of bricks and mortar retailers over the past decade, the vast majority of consumers (92 percent according to Forrester Research) still do their shopping in-store. But with the Internet providing a wealth of product information in the form of reviews and comparisons, as well as special offers and promotions, more and more shoppers are browsing their mobile devices in store. Now researchers at IBM Labs are looking to make it easier for shoppers to get instant product details and deals in-store with a new augmented reality (AR) shopping app.

The app, which is being developed by IBM’s Research lab in Haifa, Israel, doesn’t rely on barcodes or RFID tags to recognize products, but instead uses the camera of a mobile device and compares the image to a database of product packaging. The image processing algorithms combine techniques used in facial recognition, color and shape matching, and associations with surrounding products. The app also takes into account the camera angle and distance from the product to help distinguish between products.

Once the product is recognized, the app will then overlay digital details of the product on the image. This can include nutritional information, price, reviews and discounts on offer at that time. Consumers can also choose to opt into a social networking feature that would see comments or reviews from friends and family about a particular product added to the information displayed.

“In the age of social media, consumer expectations are soaring and people want information and advice about the products they’re going to buy,” said Sima Nadler, Retail Lead, IBM Research. “By closing the gap between the online and in-store shopping experience, marketers can appeal to the individual needs of consumers and keep them coming back.”

When setting up the app, consumers create a profile and detail product attributes that are relevant or desirable to them. These include dietary preferences (allergies or a preference for low fat products, for example), pricing, environmental (biodegradable packaging, etc) and religious preferences. Then, when the shopper pans across a shelf of similar products, such as breakfast cereals, the app will indicate which ones meet the customer's criteria.

"The idea of standing in an aisle in the supermarket and having your mobile device point out the gluten-free cookies you need can be a real time saver," said Amnon Ribak, project leader for the augmented shopping app. "This has the potential to completely change the shopping experience from one of hunting, reading, and searching to simply picking up those products you prefer."

While shoppers get instant access to the wealth of product information available online, retailers would also benefit. They can use the app to attract customers back with the use of loyalty programs and digital coupons, while also gaining an understanding of customer preferences that would enable them to suggest related products in other aisles. IBM says that because such suggestions would be tailored to individual customer preferences and occur in-store, they would not be seen as intrusive or annoying. The app would also help retailers optimize floor plans and the location of products in-store.

The augmented reality app is currently still at the prototype stage, but expect it to be merging the online and offline shopping experience in the not too distant future.

Source: IBM Research

  • Facebook
  • Twitter
  • Flipboard
  • LinkedIn
3 comments
The Hoff
Just give me a freakin search app for the store and I'd be happy! I can't believe in this day and age that I can't walk into a major chain grocery store and ask my smartphone for directions to an item, just the isle is all I need. This is what men want, what are they waiting for?
Georges Goossens
Hello, the idea is to stow away the goods so the customer has to search. By searching the customer will stumble up on goods (s)he was not planning to buy. I once proposed the idea of navigating through the store, I was "scolded" (well not that extreme, but enough to get it reminded by myself many times) for naive ... I that is so true ;-)
The Hoff
I believe you but that marketing does not work on me, it just pisses me off and I will leave without it.