Genie, the new data layer in the Salesforce platform that makes application silos disappear

After four years as CTO of eBay and a decade prior to that leading the engineering of Salesforce’s platform and brand, Steve Fisher had settled into a comfortable retirement two years ago, enjoying a state-of-the-art theater at his oceanfront home. But just as it seemed the credits were about to roll on his career, his phone buzzed.

It was a FaceTime call ... I looked down and my phone is buzzing and there's a picture of Marc [Benioff] and I thought ‘Oh that's unusual.’

The Salesforce CEO was calling to discuss the work his company had been doing to solve one of the most pressing challenges facing large enterprises today. Their customer data is trapped in a sprawling mess of digital marketing and CRM tools that have sprouted up over the years. Fisher knew from his own experience at eBay what it takes to bring all that data together, overlay it with AI, and then meet customer expectations for a consistent, relevant experience across touchpoints that vary from email, advertising, web and mobile to call centers. He'd also been thinking that this was a platform that all businesses, not just web giants like eBay, were going to need, as digital channels become ubiquitous. Fisher says:

I just was feeling the itch that somebody needs to build that platform. And that's when Marc called. And I said, 'You know, I think we need to build this platform.' And Marc said, 'Well, we're building it, so why don't you come over and help us out?'

Benioff introduced him to the team that were working on the project and Fisher was sold. He recalls:

I thought, I've just got to do this. I can't just spend the next 30 years watching movies and sitting on the beach. I've got to go build this product. And that's what I wanted to come back to do — build this product and really bring it to life across the entire Customer 360 ...

There's lots of companies that can deliver a piece of the solution, but who else can really put it together and really put the end customer at the centre and deliver that consistent, relevant, magical experience for them? I just didn't think that anybody else other than Salesforce could really do that. And it was just too compelling for me.

A step change in the Salesforce platform

The project, now called Genie, launched at Dreamforce last week. Like several of its competitors, Salesforce had been building a Customer Data Platform (CDP) to connect data across those separate application siloes and achieve a single ‘360-degree view’ of the customer — a long-promised goal that vendors still struggle to deliver. But it then decided to build something with a broader reach, able to bring together not just everything that marketers needed to interact with customers, but extending across every aspect of CRM, encompassing sales and service too. Fisher explains:

We launched a version of this for marketers that we called the CDP. That was our opportunity to prove out the technology, test out the technology, get some customers, get some feedback — but that wasn't really the vision. We're a CRM company and our vision is to provide personalized engagement across every touchpoint, human, digital, in-store. That's basically our mission, to build customer relationships.

Genie therefore represents a step change in the way that Salesforce architects the platform on which all its applications run. He elaborates:

I think now, with Genie, it's really probably the biggest step forward we've made since back in 2004, 2005. Bringing this hyperscale, real-time data, optimized for engagement, optimized for analytics, optimized for AI — but deeply connected into the Salesforce platform.

Under the surface of Genie

Genie's been described as a data lake, which technically is accurate, but as a metaphor conveys completely the wrong impression. We think of a lake as a smooth body of tranquil water, whereas Genie is a tumult of rapid cross-currents, at the nexus of countless weirs that channel the rushing inflows and outflows of data. At a briefing last week, Fisher and two colleagues outlined the key elements that Genie brings to the Salesforce platform. There's a lot of underlying technology that handles the massive data flows — my notes mention a "big data hyperscale infrastructure" — but what's really important is that Genie is a new layer of abstraction that normalizes data from multiple sources so that it can be analyzed and acted upon as a single dataset.

Data can come from any source, whether it's any of the Salesforce applications, from web or mobile interactions, or via an API from other enterprise applications and data stores, and it's then turned into a Salesforce data object by applying any of over fifty extensible data models, or customer graphs, which cover entities from people and accounts to consent, loyalty, security and so on. These out-of-the-box data models can be tuned to specific requirements and they can apply fuzzy matching with probabilistic scoring to help automate the process of ingesting data. There's an identity resolution tool that automatically joins up records across different IDs for the same individual. Muralidhar Krishnaprasad, EVP Software Engineering, who leads the Genie project, explains why these graphs are important:

The power here is this. These are virtual models. That means even though your data is coming from a hundred different sources, you can logically map to this model so that the rest of your applications don't have to worry about what your schema was.

Working on this normalized data, it's then possible to view standardized segments, dashboards and calculated insights or trigger actions such as business flows, customer journeys, or the creation of reports in Tableau. For example, Health Cloud is now shipping a unified health score that aggregates all the device feeds from customers or patients. In another example, marketers can create segments and then personalize experiences or commerce journeys.

Genie has a multi-layer structure to accommodate data sources with different cadences, ranging from real-time to batch, and a real-time cache layer is able to respond to requests within milliseconds, which is essential for certain customer-facing applications. Administrators can specify dataspaces with restricted access to a slice of data, which allows specific teams or business units to work on their own projects without impacting others. Data can also be exposed into Snowflake without copying or into AWS Sagemaker to create complex ML models.

Playing in three CRM product segments

Another way of looking at this is that Salesforce has broken down the silos that separate data between applications and built a common data layer that not only feeds all its CRM applications but is also available to machine learning, AI and analytics. Effectively, the data has become 'headless' although Salesforce doesn't align itself with the new generation of composable commerce and content vendors that take a more best-of-breed ecosystem approach to CRM.

Curious about where Genie is taking Salesforce, I asked David Schmaier, President and Chief Product Officer at Salesforce, for his take on the trend towards composable applications. His response was that Salesforce views the CRM market as falling into three segments based on product architectures. The first of these is the declarative CRM market. He explains:

We're by far the leaders in the declarative CRM market, declarative meaning it comes out of the box, and you can use clicks not code to tailor it to the way your business works, and we even go further than that with industries. We're bigger than the sum of everybody else in that space.

There's a self-subscribe CRM market, a product-led segement which targets small businesses with ready-to-run, packaged solutions. Salesforce plays here too, as he explains:

That's where we launched Salesforce Easy, which is really for SMBs, sort of a simple C360 Suite. We took it from 27 clicks to sign up for Salesforce to three. You put in your email, you put in your credit card, and bam, you're in. And then you can import your contacts and you're off and running. And we think this is really cool. We have a whole Easy initiative just to make everything easier and simpler.

The other segment is the API-first CRM market, characterized by headless commerce and content vendors, in which best-of-breed components are co-ordinated via universal API management. Salesforce is the leader here too, he says, quoting a recent study by API platform vendor Postman which finds Salesforce is by far the leading source of APIs that businesses call. But while Salesforce is able to support a composable approach, he sees this as a smaller market, limited by the challenges of co-ordinating this kind of architecture. But Salesforce still accommodates customers that want to take this approach, as he explains:

Even though we sometimes lead with the declarative approach, if somebody really asks for a composable headless architecture, we listen, we don't mandate. The customer's really always right.

My take

Don't underestimate the importance of Genie. It's a major upgrade to the data layer within the Salesforce platform, and a fundamental break with the traditional vertically-stacked architecture of enterprise applications. This has been forced by the need to deliver data instantly wherever it's needed, whether that's in customer interactions where, as Schmaier likes to emphasize, "milliseconds matter", or to expose ever-larger datasets to machine learning and data analytics. This is the inexorable pressure of the move to Frictionless Enterprise with its emphasis on real-time response and on-demand resources.

The next question this raises in my mind is whether Genie represents Salesforce moving towards what I've started calling a Tierless Architecture, in which data becomes an API-first resource that's available to any touchpoint or service. I find Schmaier's response here interesting. Salesforce still leads with what he calls a declarative product, in which the application comes ready-built but with huge choice in how it's configured. He sees that as distinct from the composable market, where the engagement layer is more flexible and therefore requires a greater level of technical resource to deliver a finished application. I suspect the distinction is more a matter of degree. The composable vendors are on track to provide more toolkits and templates to get closer to an out-of-the-box offering, while Salesforce is providing more and more flexibility in how the application is structured. Genie may be a pivotal moment towards the convergence of these two approaches, and I expect to see continuing pressure on Salesforce to maintain its momentum towards a more composable architecture.