Welcome to Confluent - understanding ‘data in motion’

Derek du Preez Profile picture for user ddpreez September 15, 2021
diginomica welcomes Confluent to our partner family at a time when understanding streaming data in the enterprise has never been more critical.


Created inside LinkedIn a decade ago, and spun out as an independent business in 2014, Confluent has since scaled to become a multi-million dollar company and also had a very successful IPO earlier this year.  

Built on one of the industry's most successful open source projects - Apache Kafka - Confluent has one primary aim: helping enterprises understand their ‘data in motion'. But, what does that mean? When I spoke recently with CEO Jay Kreps, he explained: 

The idea behind the technology was to focus on data in motion. Basically, there are all these databases that companies have spent a tonne of commercial investment on, intellectual effort, but that's really all about the problem of storing data. How does it sit? How do you look up little bits of it when you need it? 

That's the domain of databases - hugely successful, hugely important. But in a modern company, it's not like you just have one database with everything in it. There's thousands of applications and data systems and SaaS layers. 

So what you really have is little piles of data all over an organization - for the operations of the company, for the customer experience, that all has to come together into something that's holistic and smart, and which knows about everything going on everywhere. That problem we felt like wasn't really being solved.

This is an age-old problem for the enterprise. Knowing that data can be a competitive advantage doesn't always translate into using that data effectively. In fact, it rarely does. 

But as organizations continue to migrate to the cloud and adopt microservice architectures, it's a problem that's going to become even more difficult to solve - as the data is no longer stagnant, sitting in databases. It's constantly streaming.

Part of this is culture, organizational change and skills - breaking down those departmental silos and getting teams to collaborate on data has never been easy. But equally, it's a technology problem and companies need new platforms to understand this flowing data in real-time. 

And it's a challenge that has never been more critical to organizations, especially in the wake of the COVID-19 pandemic, as optimizing operations and improving customer experience become increasingly important to survival and growth. Data is key to that and Confluent is betting that buyers both see the opportunity and need support in getting to grips with their ‘data in motion'. 

As evidence, the company this week announced a new governance suite for streaming data, offered as a managed cloud solution, allowing distributed teams to easily access the data that flows through their business. The key to what Confluent is pursuing here is the creation of a platform that ties together disparate applications, systems and teams, with data being the backbone, and making it easier to gain insights. 

Confluent sees data as the ‘central nervous system' of an organization and is looking to offer buyers a way to work together around that. 

We spoke to SOCAR Malaysia recently, a car sharing platform, which showcased exactly this. As SOCAR moved towards a microservices architecture it adopted Confluent to allow teams to work together to get real-time insights into how customers are using its services, for the purpose of improving customer experience and pursuing personalization. CTO of SOCAR, Mustafa Zaidi, said: 

One thing that Confluent and Kafka helped enable for us is the ability to stream events from our apps. How much time are users spending on a screen? What cars are they viewing? You're reviewing one car but booking another, why is that? Traditionally we have been very focused on the final transaction, when the user made a booking. 

And of course we can do a lot with that. But now we're taking this to the next level of really analyzing user behaviour in-app. And honestly we are literally just scraping the surface of this right now. That actually leads us towards greater personalization.

We will be speaking to more Confluent customers in the coming months, which will be key to understanding how ‘data in motion' is being tackled by companies. And Confluent of course will be giving its own view with its upcoming partner content on diginomica - keep your eyes peeled for the first piece in the coming days. 

In the meantime, welcome to Confluent! 

A grey colored placeholder image