It’s a tough nut to crack, and data science can help, but the limitations of traditional data warehousing technologies often hamper the process, says Kelly:
By the time you’ve formulated the question you need to ask, run your query and got back your response, the world’s already moved on.
In other words, while Midvale, Utah-based Overstock closely tracks the interactions that consumers initiate on its site, successfully converting clicks into sales is a matter of speed. And increasingly, the retailer is looking to understand how it can better perform that conversion trick not by bundling customers into broadly similar groups based on behaviour and demographics, but in a one-to-one, individualized way. It’s all about precision.
Ad hoc data science jobs
This was the thinking behind Overstock’s decision to implement a cloud-based data warehouse from Snowflake earlier this year. While the company’s traditional data warehouse is great for running traditional aggregated business intelligence reports, says Kelly, it’s just not suited for ad hoc data science jobs that often require massive amounts of compute resources on the fly:
Snowflake eliminates that problem with elastic compute resources, so when we know we have a data science job coming up that has very low aggregation and high degrees of personalization, we can go in and spin up new clusters on demand. That’s changed our data science world, because analysts can tick off a query that might otherwise take days to set up and run, or might get killed off simply because it’ll take up too many resources and is considered a lower-priority job.
Today, we can get an answer to a first question quickly and then move on to asking the second, third and fourth questions that allow us to get a much deeper insight of a particular customer and what they’re interested in buying.
What this means is that the focus in the data science team has shifted from building data pipelines on a case-by-case basis, and more towards an iterative approach to model development. Says Overstock’s vice president of product and analytics Joe Kambeitz:
A common meme in the data science world is that data scientists spend 80 percent of their time prepping data and 20 percent of their time building models. We wanted to flip that ratio.
What kinds of questions?
So what kinds of questions is Overstock now able to ask about its customers and the best marketing tactics to win them over? Says Kelly:
It might be that we want to look at every person who’s visited our website in the past hour and left without buying. We need a view on the best way to bring them back - so is this particular customer likely to respond to a marketing email, for example, or are they a member of our loyalty club and likely to come back without prompting?
The best thing about this project, he says, is that it hasn’t required a huge data migration effort in order to start producing results:
In order to kick things off, we recognized that a lot of data science jobs are based on real time and very recent interactions, so they’re not at all dependent on historical data. That meant we were able to get the first production jobs coming out of Snowflake by mid-September, and we only started work on the system at the end of June. As we get deeper into the system and the types of queries we want to run get more complex, there will be a need for historical data, certainly, but we’ll be able to pick and choose that data based on the specific questions we want to ask.
The benefits of one-to-one marketing, meanwhile, are potentially huge for a company like Overstock, Kelly adds, which operates in a highly competitive market, often against retailers with far larger marketing budgets:
We have had the data we need to target individual customers in terms of marketing for a long while now, but what we didn’t have was the technology available that would allow us to do that in a cost-effective or time-effective way. Treating customers as individuals is something we’ve wanted to do for a very long time and the data science team is loving being able to work in this way.
So much so, he says, that he recently received an email from one member of that team, describing the change in her working practices as being like moving into first class after always having flown in coach:
She and her colleagues can work at a speed that was previously unfathomable, now that they’ve got the tools they need to deliver the best work possible to our marketing organization.