Main content

How Everseen applies AI and deep learning to Point of Sale, with a checkout-free future racing towards us

Jon Reed Profile picture for user jreed March 17, 2017
My retail reviews hit on the myths and realities of the omni-channel - now it's time for AI and predictive. First up? Everseen, an Irish upstart and pioneer in AI Point of Sale technology. But as my talk with CEO Alan O'Herlihy shows, the next retail giant Everseen is set to take on is Amazon, as in Amazon Go. The checkout-free future of retail is firmly in Everseen's sights.

Everseen POS tech in action

In my last retail review, I explored how my learnings at NRF 2017 changed my view ofthe so-called omni-channel (Omni-channel may be science fiction, but a single source of truth matters). That leaves open the impact of predictive, "AI", and personalization tech on retail.

This time around, it's Everseen, an Ireland-based company that bills itself as "AI for managing sale process integrating at the point of sale." CEO and Founder Alan O'Herlihy gave me a fast-paced rundown of how his company has become entrenched in five of the ten largest global retailers.

How did they pull it off? Get ready for this one: instead of pursing the singularity, they focused their deep learning tech on a real world pain point: lost sales at the point of sale. The retail term for this is "shrinkage," which is estimated to be a $40 billion a year problem for retailers. Everseen told me that 35 percent of shrinkage is due to fraud and errors at the point of sale (POS).

O'Herlihy told me the rate of problems actually increase with self-checkout technology. They did a study of a million retail customers, breaking down their video streams at 25 frames per second:

We're looking at the POS stream in milliseconds. With that data, we showed on a million customers that eight times more non-scans happen at the self-checkout versus the manned checkout, and actually, the value is about 20% more as well. So, if you do make a mistake at the self-checkout it tends to be something more valuable than the manned lanes. That was a revelation. (see Everseen's Point of Sale Insights Report PDF for more)

The classic way to try to address shrinkage is through exception reporting, but that only addresses five percent of Point of Sale irregularities. Everseen's goal with their customers is to decrease errors at checkout, the moment that they happen. The tech combines AI algorithms, computer vision systems and "big data."

How it works: Everseen integrates with security cameras above registers. If a product is unscanned or scanned improperly, Everseen sends an alert, including an image of the non-scanned item, to the stores' security teams via mobile device or smart watch. The staff can take the right step before the customer leaves the store. (pictured left, see a sample Everseen alert of an "incident" involving an unscanned item left in the cart. The feature image above shows a different scanning issue).

One surprise: this can be a boon to customer service. You might think this would be about cracking down on scammers, but most folks who have a bad scan want to do the right thing. Alerts provide a customer service opportunity:

It's actually enhancing customer experience because now you can find someone and say, "Look, you didn't scan that product. Let me help you." People struggle with the freaking self-checkout. It can be very confusing. I didn't go after this from a customer service point of view, I went after it from a shrinkage point of view, but the benefit to customer service is even better than the shrinkage.

Inside the algorithm - years of lab experimentation

A lot of laboratory toiling went into this result. I have a mental image of a bunch of data scientists locking themselves in a room for 5+ years. It probably wasn't quite that hardcore, but O'Herlihy says that originally their "Computer Vision" algorithms weren't working right in real-time. But they kept experimenting with their streaming POS data, trying different approaches like detecting "smudges" or shoppers' hands blocking the labels. Finally, breaking it down frame by frame, they came up with a winner:

We basically worked out what the DNA of a transaction was, and we worked out the DNA of a non-scan also.

Next up: apply deep learning and run huge volumes of scans and non-scans through the system:

With the deep learning we had false positives; we labeled those as false positives and then we made it more accurate.

A retail upstart like Everseen needs loads of attitude to compete with the behemoths. O'Herlihy sees their patents and home-grown algorithms as crucial, but he also points to their high volumes of in-store retail data for deep learning. That's data even the AI giants don't have:

If you look at Facebook and Google, and all these guys who are doing deep learning, they're dependent on data to progress. The only place they can get data is on YouTube or other publicly available data. They can't get access to this.

Beyond the checkout - look out Amazon Go

Another upstart lesson: compete with yourself. Everseen has its eyes on the same terrain as Amazon Go: checkout-free shopping. The new product, appropriately dubbed 0Line, uses video cameras and sensors to track products as customers shop. Prior to leaving the store, customers will get a receipt for their purchases on their phone. They click a red or green button indicating the tally is correct (or not), and if all's well, out they go.

Right now, 0Line is in trial use in a lucky convenience store in Ireland. Once this tech goes live in more stores, O'Herlihy thinks the dominos will start to fall:

Think about the queue management. If you're a retailer with those old school checkouts, the guy next door is going to kill you with this technology, so you need to move.

Gotta love how O'Herlihy isn't intimidated by Amazon Go:

The good thing is that with Amazon Go, no retailer is going to trust them with their information. Retailers are afraid of Amazon, they do dynamic pricing, they're aggressive, they're moving to the physical store. That's why we see it as a huge opportunity for Everseen.

Using not-quite-safe-for-work language, O'Herlihy expressed utter confidence that the checkout - and self-checkout - is going away:

If you Wikipedia the freaking checkout, it's 200 freaking years old. So we're stuck in a 200-year-old model. Then we went half-hearted with the self-check out thing, which is the biggest heap of shite ever, but whatever. This is going to change the industry.

Amazon isn't O'Herlihy's enemy, more like frenemy. He credits Amazon with legitimizing the push beyond self-checkout, resulting in an Everseen funding round:

These b!x!trds at Amazon are brilliant right, because they said "The heck with self-checkout." Now I've raised 12 million dollars for my company. If I went to VC a year ago and said, "No checkout needed," they would have sad, "Yeah, you mad Irish dude, stay on the one that's working."

But it worked out - and now we'll see. If all goes well with the big retailer, they'll roll it our from one location to about thirty stores around Dublin. Meantime, Everseen's thirty research scientists in Romania are busy evolving Everseen's solutions. That will include personalization to real-time sales and inventory, including perishables:

If we're at 3:00, and there's loads of chicken sandwiches left, and we know some guy is down around the corner, we'll be able to send him an offer, "one Euro for the chicken sandwich." Then he's in and out of the store in minutes - no checkout queue.

That's an AI I can get behind. If we're going to change the world, might as well start with chicken sandwiches.

End note: I have more predictive and personalized retail initiatives to share, which I'll cover in future installments.

Updated 7am Saturday UK time to correct a few small issues from a prior version of the article.

A grey colored placeholder image