Confluent CEO on why data in motion is now central to enterprise success

Confluent is built on one of the technology industry's most successful open source projects - Apache Kafka - which was created inside LinkedIn in 2011. Confluent was spun out in 2014 after the recognition that historical, stagnant data wasn't going to cut it for modern digital enterprises and that there may be broader commercial appeal for understanding data events in real-time, beyond the four walls of LinkedIn. 

That bet turned out to be accurate, as the company - led by CEO Jay Kreps - has since scaled to become a multi-million dollar business in its own right, which had a successful IPO back in June. Confluent's first set of results saw the company bring in $88 million in revenue in Q2, up 64% year over year and it reported that it now has 617 customers with $100,000 or greater in ARR, up 51% year over year. 

Not only that, but the company's cloud business accounted for $20 million in revenue, up 200% year over year. 

In short, Confluent's proposition of ‘data in motion' is gaining traction with buyers. And given the company has just gone public, diginomica thought it an opportune time to speak with CEO Jay Kreps about his ambitions for the company and to get a better understanding of what's driving purchasing decisions for end users. 

Speaking to the core of the Confluent's purpose, Kreps says: 

The idea behind the technology was to focus on data in motion. Basically, there's all these databases that companies have spent a tonne of commercial investment on, intellectual effort, but that's really all about the problem of storing data. How does it sit? How do you look up little bits of it when you need it? 

That's the domain of databases - hugely successful, hugely important. But in a modern company, it's not like you just have one database with everything in it. There's thousands of applications and data systems and SaaS layers. 

So what you really have is little piles of data all over an organization - for the operations of the company, for the customer experience, that all has to come together into something that's holistic and smart, and which knows about everything going on everywhere. That problem we felt like wasn't really being solved.

Kreps explains that during his time at LinkedIn it became clear that data sitting in databases all over the organization wasn't going to work in a digital environment. Instead, Apache Kafka (and as a result Confluent) was created to get to grips with the reality of data flowing in digital environments, in order to improve both operations and customer experience. 

Understanding the use case

Kreps gave the example of retail as an industry that has had to adopt data in motion at the core, given the digital demands that are being placed on retail business' operations and customer experience. 

In the past a retailer would, for example, at the end of each day take stock of what had been sold and that data would then be analyzed by HQ, which would then perhaps have an idea at the end of each week what inventory was available and what new products needed to be ordered. Only then at the end of the quarter would the retailer then perhaps consider adjustments to pricing and promotion. Kreps says this wouldn't stand up in today's world and explains: 

It was very a batch processing oriented world, where it's all about storing the data and then shipping around whatever was stored periodically. So, what's wrong with that? Well it really falls apart when you have software more directly involved in the operation of the business and in the customer experience.

The operations side of the business is focused on distribution centres, warehouses, inventory, all getting moved around. That's something that's happening in real-time, that's happening out in the world right now. 

And then on the customer experience side, if you're shopping and you want to go on your phone and see if there's something in the store down the street, well the retailer needs to know what's in the store down the street - not what's in the store last week when they last ran that inventory roll-up. 

And then you may want to buy it online for delivery, or buy it online and pick it up in-store. So that melding of the digital side of retail and the physical side is really important. Strategically, this is very important

But to do that you have to have this very continuous, fully integrated view of what's happening in the business, across digital, in the warehouses, in the stores, it's all got to come together, all the time.

Confluent's product works by connecting to enterprise databases and enterprise systems, on premise and in the cloud, allowing users to take the stream of data changes occurring across these and react to it in real-time. Another industry example that Confluent has found popularity is in the banking sector, where financial institutions are using its product to identify fraud - where relying on batch processing of historical data would be nowhere near quick enough to spot abuse of the system. Again, as one can understand, enabling this in real-time plays directly into the positive experience of banking customers. 

Fresh thinking

The COVID-19 pandemic has given many organizations pause for thought, particularly as it relates to their technology and data investments. The disruption caused by national lockdowns shone a bright light on the frailties of many enterprise systems, both in terms of operations and customer experience. 

Kreps has seen evidence of this too amongst Confluent's customers and prospects. He says: 

I do think sometimes when there is an emergency, it does cause people to think in a fresh way. For a lot of these industries, having all of their customer interaction go online and having a lot of the internal interaction be digital, really made people think about what the role of that is in the business. Long term, 10 years from now, are they really taking the right steps to get there, for instance? 

Confluent also has the benefit, according to Kreps, of appealing to the enterprise's developer community, as well as the senior executive's desire to understand their business as a whole. Driving demand at both ends of the organization should bode the company well, especially as demand for integration and understanding of data across organizations grows. Kreps says: 

From relatively early on, we had a good bottom up story, which is: how is this useful for one software engineer and getting their job done? If you're the person whose job is to go build the fraud detection system, is this making your life easier? 

But actually from very early on, even during the initial use at LinkedIn, there was also very much a big picture story about: how to share and harvest data. How do we do that in a way that's safe, secure and correct? How can we really unlock and integrate the different parts of the company? And that is a big picture story. 

How do we integrate across older systems and newer cloud capabilities? How do we bring the next generation customer experiences and better operational capabilities? And do all this without starting from scratch. I think that's right at the heart of the problem that a lot of companies are trying to solve in technology. And that's why I think we see engagement at a senior level too, it kind of lines up to that aspect of what companies are going through. 

My take

Now more than ever, when talking to buyers they are thinking about how they can make better use of their enterprise data. Confluent's core proposition is inextricably linked to the two key challenges we've heard a lot about during the COVID-19 pandemic - improving operations and customer experience. By commercializing Apache Kafka, and offering a product that connects the dots across a company - both for legacy and modern systems, with real-time data insights being the end goal - Confluent's offer is timely and compelling. We look forward to speaking to some customers too, to get a deeper understanding of how this has played out, because as we all know, data projects are never straightforward or easy. 

For Kreps, the focus post-IPO is continuing to build a company that has the right people that can continue to build a leading product. Kreps is well aware that there is still room for growth. He finishes our conversation by saying: 

The biggest challenge for a company at our stage is always execution, right? It's a little bit mundane but it's actually quite hard. We're lucky enough to be a first mover in a space that's really caught on - but we're not the central nervous system of every company in the world yet. So there's a lot of work left to do. 

I think a lot of that comes down to building the right product, working with customers in a way that really helps them get the value out of it, scaling in a way that builds a company that's really great, not just big. And I think it's really hard to do all those things at the same time.