Deliveroo takes away hot business insights with Snowflake

With the arrival last year of Amazon Restaurants and UberEats, the UK’s food-delivery services market is hotting up. That has big implications for incumbent Deliveroo, which in April this year announced plans to ramp up operations and, to some extent, ditch its asset-light model, with the introduction of Deliveroo Editions.

At the heart of this new offering is a network of pop-up kitchens that host teams from partner restaurants and act as hubs for food deliveries. In other words, in order to stand the heat, Deliveroo’s getting into more kitchens.

As the company puts it, Deliveroo Editions “will put an end to postcode food envy.” What that means in practice is locating pop-ups in areas where the company has identified high demand for particular types of food and insufficient supply. In order to find the most potentially lucrative hotspots, it’s turning to data analytics, explains Henry Crawford, head of business intelligence at Deliveroo:

This is not something you can leave to guesswork. To decide where new Deliveroo Editions pop-ups should go, we’ll look at a wide range of internal and external data.

Internal data will show us what food is already available in that area and what customers in that area like to order. External data will give us insight into local demographics, local traffic conditions. All this will feed into the process of finding a suitable location, and we’ll also provide the Editions team with a prediction on how that unit might be expected to perform, also based on data.

Using really broad datasets that we can explore with a bit of freedom means that we can answer what might otherwise be a pretty broad question from our CEO: ‘Where should we put Deliveroo Editions next?’

Snowflake doesn’t melt under pressure

At the heart of this effort, as well as other analytical processes at Deliveroo, is technology from cloud data warehouse provider Snowflake. Currently holding some 35TB of data (up from 7TB in May), this warehouse was implemented when the company ran into problems with its previous, AWS Redshift-based data warehouse.

The issue there was concurrency, Crawford explains; around nine out of ten of Deliveroo’s 1,500 staff, he reckons, need at least some access to reports, but AWS Redshift was struggling to cope when hundreds of them needed that access at the same time:

We like Redshift, it’s the right tool in a lot of situations, but on a Monday morning, for example, things were just running a lot more slowly than they should do. What we like about Snowflake is that it elastically scales up and down, so if we have a peak for three hours on a particular day, we can just ramp up the number of clusters, but we don’t have to pay for that reserved instance during the rest of the week.

Given that Snowflake is also based on AWS cloud infrastructure, the migration was pretty straightforward, he says. The process took between one and two months, with the most time spent on QA and auditing of the new data warehouse, rather than the transfer of data itself.

Data-driven decisions

With Snowflake in place, Deliveroo staff can now run many hundreds of concurrent queries and, importantly, they’re not asking data engineers on Crawford’s team to help them figure out what the pipeline of queries looks like, nor where their own query sits in the queue. They’re simply getting on with the business of making data-driven decisions, Crawford says.

Whereas previously, business analysts might spend a lot of time waiting for results, they now have time to really think about business problems, accumulate multiple reports in search of answers, hypothesize and retest their theories based on the most recently available data. They have the space and rapid answers they need to keep their thought processes running.

In turn, data engineers are freed up to focus on building new models, tools and algorithms to help the company answer its most burning business questions. A good example is ‘Frank’. This is Deliveroo’s driver despatch engine, a machine learning system that constantly calculates and recalculates how to best match available delivery people to orders, based on data relating to routes, food preparation times, real-time traffic conditions and weather. Says Crawford:

We have a very talented data engineering team that’s always on the look-out for new technologies and new approaches. The database world moves pretty quickly, so it’s a challenge to keep up-to-date on what’s available. But we had a significant need to scale up and down, we didn’t want to host infrastructure ourselves and we were looking for a fast migration – so Snowflake ticked all those boxes for us.