SOCAR Malaysia drives towards in-app personalization with Confluent

Profile picture for user ddpreez By Derek du Preez September 7, 2021 Audio mode
Summary:
Car sharing platform SOCAR Malaysia has adopted Confluent to create an event driven architecture, where it has a better understanding of its users in-app.

Image of a SOCAR Malaysia car
(Image sourced via SOCAR Malaysia website)

SOCAR Malaysia is looking to disrupt the model of car ownership in the peninsular of Malaysia (and has recently expanded into Indonesia), whereby through the use of a mobile app, users can sign up, book a car, unlock it and drive away. It also has another platform that allows rental companies or private car owners to list their vehicles on the app for hire too. 

Central to SOCAR's ambitions is its approach to data use and analytics. Speaking with Chief Technology Officer (CTO), Mustafa Zaidi, it's clear that the company has a mandate from the top to build a culture of widespread data use amongst employees, where they're encouraged to investigate and analyze - with the aim of improving customer experience.

However, in pursuit of getting a better understanding of how customers use their platforms, in recent months SOCAR has moved away from a monolithic database and adopted a microservices architecture running out of AWS. The company has then also implemented Confluent, which is built on Apache Kafka, and allows for distributed event streaming - meaning SOCAR can get real-time insight into changes happening in-app. 

The end result being that SOCAR employees have a clearer, real-time understanding of how its users are interacting with its platforms, allowing it to respond effectively and build in more in-app

Speaking to SOCAR's approach to data, CTO Zaidi says: 

I think in some ways, some of the things that we do with data are a little bit different than what you might find in a corporate environment. In our case, while we do have, for example, a data analytics team, there is an expectation from higher management in the company that almost anyone in SOCAR should be able to access relevant data and analyze it. 

We have a very, very large set of users who have access into our data warehouse and are regularly writing and running queries, building dashboards and doing analytics. 

So, we definitely have a very open data model. And honestly, it's been even a little bit of culture shock for people coming into the company, being surprised as to how open we are in terms of allowing access to data for our employees. All of our employees are very comfortable with analysing data on an ongoing basis. 

Unlocking real-time pipelines

SOCAR employees had real-time access to data when the company was using a monolithic database and there was an expectation that they would not have to put up with a multi-hour ETL delay at any point. So when the company moved to a microservices architecture, it needed a platform that would continue to meet this expectation. 

But with multiple individual microservices running in the background, users are unable to access multiple databases in real-time to run a query. This is where the use of Confluent became necessary. Zaidi explains: 

Our use of Confluent started from an analytics perspective. SOCAR wants to be on the ball in terms of how transactions are ongoing on our platforms at any point in time. That was the requirement: how can we have a data warehouse that is effectively real-time streaming data from our databases for analytics purposes? This was the challenge that Confluent helped us overcome. 

The use of Confluent will allow SOCAR to shift towards an event-driven architecture, which means that it streams data consistently and allows for data users to analyze any changes. Zaidi adds: 

From a data warehousing perspective and live change, data, capture, it's fully implemented and any new service that we have, we have a process that we make sure that we build those pipelines and incorporate them into our data warehouse. 

At the same time, the second part that we actually are going to be kicking off is to create an event driven architecture. We've had multiple use cases where we believe that having directly synchronous or asynchronous API calls is not the way to go, we really need a message queue to be able to handle the latency aspects. 

I hope that before this year is over we will have multiple implementations of event driven architecture using Confluent.

The customer journey

How this will improve the experience for customers is best illustrated by an example. For instance, if a user books a car using the SOCAR Malaysia app and they want the car delivered to their home, but for some reason there's an issue with the delivery service, that would prevent the customer from completing the booking. SOCAR doesn't want that, both in terms of its booking rate and improving customer experience. And so the platform enables for all the delivery requests to be put in the messaging queue, which will be processed effectively using the message brokers and Confluent, identifying any problems. 

SOCAR is aiming to move all of its transactions to an event driven and consistency based architecture, wherever it can. In terms of how this will enable personalization down the road, Zaidi explains: 

One thing that Confluent and Kafka helped enable for us is the ability to stream events from our apps. How much time are users spending on a screen? What cars are they viewing? You're reviewing one car but booking another, why is that? Traditionally we have been very focused on the final transaction, when the user made a booking. 

And of course we can do a lot with that. But now we're taking this to the next level of really analyzing user behaviour in-app. And honestly we are literally just scraping the surface of this right now. That actually leads us towards greater personalization. 

One of the early use cases we're looking at is when a user looks at a more expensive car but it is not within their budget today, so they eventually book a less expensive car model. Can we encourage the user to take that next step by issuing a promo that's custom to them in real-time? We're just on the cusp of this and we're starting to evaluate and figure out how we can make this happen by looking at this clickstream data that has been enabled using the Confluent hosted Kafka platform.