Hungry for data - HUEL decouples infrastructure with Confluent

Derek du Preez Profile picture for user ddpreez March 26, 2024
Summary:
Fast-growing subscription business, HUEL, delivers ‘nutritionally complete food’ to its customers and has migrated away from a range of legacy systems to Confluent to better support its future business needs.

An image of a range of HUEL products
(Image sourced via HUEL website)

HUEL is a fast-growing food subscription service that is increasingly popular amongst health-conscious consumers. It has built a name for itself by delivering a ‘nutritionally complete food’, where every HUEL meal contains a balance of 26 essential vitamins and minerals, protein, essential fats, carbs, fibre and phytonutrients, in a single product. Launched in June 2015 by Julian Hearn, HUEL has grown to include a wide range of meal subscriptions that range from pre-made drinks, to ‘mix it yourself’ powders, as well as snack bars - catering to people that are health conscious, but short on time. 

In its Full-Year 2023 results, HUEL hit £185 million in revenue, off the back of eight years of continuous growth, and has now sold 300 million meals to 80 countries worldwide. Anyone that attends the gym regularly in the UK, or is familiar with the health market, will almost certainly recognize the brand’s identifiable black and white logo. 

However, like many start-ups, HUEL began by bundling together a variety of different technology services, to enable its customers to discover its products, build subscriptions, and communicate with the company. This legacy environment was holding HUEL back not only from the scale it was looking to achieve, but also from getting the insights it needed from its data to provide customers with the superior service it is aiming for.

With this in mind, HUEL began a journey of looking to decouple its infrastructure, moving towards an event-driven, microservices-based architecture, so that the business could offer its customers more choice, so its warehouse could operate more efficiently, and so that subscriptions could scale effectively. 

diginomica spoke with Chris Traverse, Director of Engineering at HUEL, at last week’s Kafka Summit London, where he explained why the company needed to migrate away from the “monoliths” that the company had been operating with, towards a Confluent environment. Commenting on the need for Kafka, Traverse said: 

I've basically been at HUEL since the start. I was the first person in the company in tech. I think I’m the eighth longest person there. I came in, built some websites, built some systems, and then suddenly we’ve got all these plates spinning, post-it notes sticking to things, nothing is any sort of sensible system. 

Slowly over the last couple of years, we've been bringing in people who know better than I did about what we were supposed to be doing for those things.

Subscriptions were the biggest driver for change at HUEL, which is fundamentally a subscriptions-based business. The company had been using a partner plug-in on its Shopify platform, but it wasn’t suited to HUEL’s varied product offering. Traverse said: 

We'd worked with several companies, trying to figure out how we could do this within the Shopify environment, but we couldn't really get to anywhere that works. 

At that time HUEL only had three people in its engineering team, including Traverse. He said that the team came to the realization that what HUEL needed was to bring its technology environment entirely in-house, to make it run how it needed to for customers. He said: 

It got to the point where we had just built so many of those things ourselves, it made sense to build the other 40% and take the whole thing in house. 

And then looking at that, it was a case of: what's the best way to do this? We don't want to just build that other 40% on top of what we've got, because what we've got is built on top of someone else's stuff anyway. Maybe that's not what's gonna work best for us as a business moving forward. 

So it was literally a root and branch tear down of what we were doing, why we were doing it, what we saw the problems being going forward and then looking at the technologies to help us fix those problems, smarter and faster. 

An event-driven approach

HUEL has a variety of different flavours, across a number of different products, which means customers can mix together forty or fifty different ‘skews’ to create their own customized offering. Keeping that data in a way that turns up in the warehouses, that makes sense to those teams and the business as a whole, was proving too difficult on the Shopify platform. This is what led Traverse and the engineering team to consider Kafka and an event-driven platform, which decouples the data and systems into microservices. He said: 

We had to make it turn up in a way to fit all of their systems. And then looking at event sourcing, it was just a bit of a light bulb moment for us. 

Yordan Dobrev, Head of Backend Engineering at HUEL, was brought on board to implement Kafka for the company and has been with the business for three years now. He explained: 

Everything is Kafka for us at the moment. It is the backbone of our entire infrastructure. Everything is completely decoupled. Everything is completely separate. And everything communicates only through Kafka. 

What we needed was the decoupling of services to enable the rest of the business. That was the big blocker for the business moving forward: having that analytical side, that empowers the business to make decisions. Before, everything was a plugin or integration for the data teams. 

For prediction and inventory management, we can now actually drop orders when we think it is best for us, best for the warehouse, best for the customer in terms of cost. 

Dobrev said that HUEL needed to opt for an event sourcing platform because the biggest blocker for the business was having the engineering and data analytics sides of the business so tightly coupled. In order for the business to get insights into what is and isn’t working, these had to be separated out into microservices that include subscriptions, notifications, orders, products and discounts. Dobrev explained: 

We evaluated a lot of different solutions, but Confluent worked perfectly. And the support we got from them, from the account management team, from the engineering team, really enabled us to put our heads together and decipher what was best for the business. 

If you go the event-driven way, the event sourcing way, it is an eye opener. It is a safety net, because we know whatever happens, we have that data that's immutable. 

Even if something happens, or we accidentally delete something, or we have an issue with some service, we can always go back in time, replay everything, look at the whole process, and we can fix the data, and we can fix our systems. We can get to a point where it's stable, so we never really have that worry now that we’re going to lose everything.

HUEL now only has less than 5 per cent of its legacy environment in operation and this is the last quarter that elements of it will still be in play, before the company moves entirely to a full Confluent infrastructure. As Dobrev said, HUEL has almost “killed” everything that was legacy. 

For Traverse, this project has been entirely about enabling the business. And the new Kafka environment should better support the business in its ambitions going forward, using data as the key enabler. He said: 

We were lucky that the business has confidence in us that we know what we're doing, we didn't really have to fight very hard to get business buy-in on this. I spoke to a lot of people this week that have asked ‘How did you get the buy-in for this technology?’. You're not getting buy-in on the technology, you're getting buy-in on the way to solve the problem your business has. The technology could be anything, the problem is what the business is interested in. 

We've got our subscription platform up and running. And it's doing everything the old system used to do, but better, smarter, faster. That allows us to iterate from there. We can really look at all the data we've got from every interaction anyone's had with that system over the last year, or year and a half, and look at what customers are trying to do. 

Loading
A grey colored placeholder image