Already deployed in over 50 stores around Japan, the VaakEye system constantly monitors security camera footage, detects suspicious activity and alerts staff, who can instantly review the footage and act on it. And the company is getting ready to launch Amazon-style auto-checkout as well.
Vaak is the third startup company for Ryo Tanaka, who spoke to New Atlas over the phone from his Tokyo office and said that his key driver was to make a difference. "I wanted to make a social contribution on a global scale," he told us. "There are many data management companies, but no companies to manage unstructured data. My first insight was that in order to build an unstructured data management company, I'd focus on human behavior through video images."
VaakEye, the company's first product, was launched in March as a subscription service. Costing US$162 a month per security camera in your store, it's a cloud-based deep learning AI system that constantly monitors the footage coming in, automatically detecting "suspicious behaviors" and instantly sending alerts to store employees.
The system has been trained on more than 100,000 hours of footage, both using actors to set up crime situations and going through massive troves of real-life security camera data. What's it looking for? "Action related to criminals," says Tanaka. "Looking around anxiously, searching for cameras, moving a little bit faster than normal."
It can also see when items from the shelves are pocketed or slipped into a bag. Each behavior the system notices is weighted for suspiciousness, and once a customer reaches a certain threshold, the system sends its alert, complete with video clips, to the appropriate staff member.
"The system is running now in 50 stores in Japan," says Tanaka, "and we're preparing to launch globally. Right now it's 81 percent accurate in detecting crimes using experimental data."
VaakEye analyzes human movement at more than 100 points across the body, and the company believes it will soon be capable of detecting other kinds of antisocial behavior as well, including physical assaults and more complex motions. One can easily see this kind of technology being applied to areas like London, whose multitudinous security cameras are constantly watching, but require a human in the loop to "see" without this kind of tech
VaakPay – checkout-free shopping
Vaak's next project, due to launch in August, extends similar technology further. Using the same streams of security camera footage, it watches what customers take from the shelves and gives them a completely checkout-free shopping experience, much like the Amazon Go uber-convenience store.
Instead of an Amazon account, customers will have to set up a VaakPay account on their mobile devices and click a button to let the store know they're in there shopping. The system will track what they pick up and put down, and automatically charge them for what they walk out with, without anyone needing to step up to a counter.
The system will also be able to work for non-VaakEye account holders, "if we can provide the cash register," says Tanaka. In this case, you'd bring your shopping up to the counter and simply pay for it, the system having tracked what you've got. So even if you're paying cash, it helps to reduce your wait at the register.
It'll be interesting to see how this kind of technology is greeted as it spreads across the world. Certainly, knowing you're being watched by AI systems could raise a number of privacy issues given the efficacy of facial recognition systems already in use, but Tanaka claims his goals are more idealistic.
"Our vision is a convenient world without crime or accidents," he says.
Source: Vaak
(One friend who worked in retail reported, in fact, that a super-low "shrinkage" rate meant that the displays weren't enticing customers to want the products at all.)