Headless meets serverless - a tierless architecture for frictionless enterprise

Composable architectures have been the holy grail of distributed computing since the very beginning of networking. The ideal is to have each resource on the network do what it does best, and communicate seamlessly with every other participant. Get this technology architecture right, and the implications for how businesses operate are far-reaching — but we've been through several false starts on this journey. There was CORBA in the 1990s and service-oriented architecture in the 2000s, followed in the 2010s by a Cambrian explosion of cloud, mobile and containerization. Only now in the 2020s does it finally seem to be falling into place, as headless and serverless architectures converge.

The latest developments are unfolding both at the presentation layer (the 'head') and at the underlying services layer (the 'servers'). Like many disruptive technologies in their early phases (think 'horseless carriage' and 'wireless receiver'), these two trends are named for what they replace rather than for what they bring. Their combined impact not only ignites huge changes in enterprise computing, but ultimately reshapes the enterprise itself. While this article is mainly focused on emerging technology patterns, these are important only because they enable a new, more connected, agile and composable approach to business operations and customer engagement.

• Today's emergent systems are headless because the presentation layer isn't fixed and therefore the user experience can take many different forms. A commerce system can be experienced on the web, on mobile, or through in-store gadgetry, while an enterprise application might be delivered as a web app, a mobile app, or within a messaging platform such as Slack or Microsoft Teams. Rather than headless, they are many-headed, with unlimited choice as to how to present the user experience.
• The underlying systems are serverless because the servers on which all of the computing runs are hidden away behind a layer of application programming interfaces (APIs). An infinite variety of interchangeable resources is available through this API services layer, ranging from custom microservices built by an organization's in-house IT team, to serverless functions delivered from cloud providers, to complete SaaS applications, and much more besides. Instead of being limited to what you are able to build and provision from your own servers, there is a global network of on-demand services at your disposal.

As these connected digital technologies mesh together, they begin to reshape the nature of the enterprise, opening up new ways to collaborate, connect and do business. We are still at the very beginning of adjusting to what this means for how we live and work. But the emergence of what we at diginomica call frictionless enterprise has become ever more needed, as we adapt our use of digital technology to span barriers of distance, timezones and now quarantine too.

This is not going to happen overnight. Like the switch from on-premise client-server computing to multi-tenant cloud services — which itself is still ongoing — the progress of the new headless-serverless paradigm will be slow and arduous. These new technology patterns require IT professionals and developers to abandon familiar, trusted ways of working. Their novel approaches are less well documented and therefore often appear less effective at first glance. Established vendors whose products cannot adapt to the new paradigm will stoke skepticism about its claimed advantages. There are many arguments and debates ahead. But there are already enough separate examples of this new architecture taking shape to signal that this is a broadly-based trend that cannot be ignored.

Talking heads - the rise of conversational computing

The first inkling that enterprise applications might go headless came several years ago with the emergence of voice assistants, chatbots and messaging apps. For the first time, it became possible to access the functionality of an enterprise application by conversing with an intelligent software agent. There was no longer any need to open up the user interface ('the head') of the underlying application, because the agent could do that for you and return the response in the same conversation. This has now become so commonplace that many users of messaging apps such as Slack and Microsoft Teams take it for granted. Low-code and no-code automations have become an everyday tool for building processes that connect across multiple applications and information sources.

Once the conversational layer becomes separate from the individual underlying applications, there are other services that can be applied to augment the user experience, often using artificial intelligence. These can range from bringing relevant knowledge to the user's attention to suggesting new lines of conversation. Other vendors are looking at examining patterns of work across different interactions to discover which are most effective. Meanwhile, some application vendors are fighting a rearguard action to bring the conversation layer back into their own domain rather than lose users to other platforms.

Best-of-breed - the emergence of business systems

Enterprise technology choices used to be more or less permanent — once an organization had installed a system and spent months getting it running, it was best left alone. Today's born-in-the-cloud SaaS applications and data sources have changed this calculation. The ease with which it's possible to start using new applications and plug in data and processes from other sources — often using a conversational layer to connect across multiple apps — has led to a best-of-breed explosion. Some enterprises run their business operations on hundreds of separate SaaS applications and cloud services, and routinely swap out as many as fifth of them every year as their needs and market choices evolve.

A new breed of IT specialist called the business systems professional has emerged to manage this fast-moving, interconnected, best-of-breed IT landscape. As I wrote last year when attending the first conference dedicated to this new movement, they are the product of three converging shifts in enterprise IT:

... the mainstreaming of SaaS for application delivery, the rise of agile and DevOps approaches to IT delivery, and customer demands for increasing agility and responsiveness in business delivery.

A year on, the conference meets again later today, this time virtually. Kristine Colosimo, who as Director of Marketing and Comms at supporting vendor Workato has been closely involved with the steering committee organizing the agenda, says that many of the participants have seen their roles grow in importance in the past year:

The scope of business technology is expanding as these people are moving up in the world. You're going to keep seeing that their teams are expanding, they're structuring them, and then they're expanding what they're doing.

It's going to be an interesting delineation between what is the VP of business technology doing versus a CIO. Do they eventually go to be a CIO? Or maybe they become something else.

Many of the practitioners on the agenda work for the digital-native Silicon Valley in-crowd, who you may argue have a vested interest in advancing a best-of-breed model — Atlassian, Dropbox, GitLab, Slack, Upwork and Zendesk, among others. But other participants are from more traditional industries such as energy infrastructure and aircraft leasing. And — just as last year's event included sessions on streamlining HR processes, accelerating revenue operations and automating financial reconciliation — this year's agenda is highly focused on how business systems teams have been delivering practical business outcomes tailored to the current climate, says Workato CEO Vijay Tella:

Roadmapping and streamlining is a big topic right now with business systems. It's not just rationalizing — streamlining for efficiency, speed, cohesion and coherence — but also the cost savings [are] definitely top of mind.

The headless vanguard - MACH, JAMstack and APIfication

A separate but equally fast-growing trend comes a little lower down the technology stack, led by developers taking a new approach to building more flexible websites and mobile apps. A few weeks ago saw the launch of a new advocacy group called the MACH Alliance, where MACH stands for Microservices-based, API-first, Cloud-native SaaS and Headless. Led by vendors and consultancies who specialize in headless content and commerce in the digital experience platform (DXP) sector, the group believes its message applies to a much broader range of enterprise applications, from marketing automation to customer service and beyond.

The headless philosophy got started early in web development due to the rise of a new generation of 'static' websites that are compatible with the continuous integration/continuous delivery (CI/CD) pipelines favored in DevOps. Mathias Biilmann, cofounder of Netlify, an early proponent, coined the term JAMstack for this type of website, where JAM stands for JavaScript, APIs and Markup. The JAMstack architecture is both headless and serverless, using APIs to fetch the content and any dynamic functionality into as many separate guises of the site as you like.

Despite this different pedigree, the MACH Alliance founders talk in similar terms to those in the business systems movement. Speaking in the MACH Alliance's launch webinar last month (which I joined as moderator), Neha Sampat, CEO of headless CMS vendor Contentstack, spoke of the ability to swap out different providers as a key benefit that customers find in the model:

They use what they consider best-of-breed today, but two months from now, six months from now, a year from now, they can change the way that their solution is architected — without being disruptive, and doing it quickly and in a way that enables and empowers their team.

Both approaches depend on APIs that are built to interoperate. You can understand why that comes naturally to the JAMstack crowd, or to those already standardized on cloud-native SaaS applications and platforms. It's perhaps more challenging for those with a mixed IT landscape. Another participant in the MACH Alliance webinar was Elaina Shekhter, CMO at digital agency EPAM, who emphasized the importance of having a suitable API infrastructure in place:

You're in the business of creating an API economy inside your enterprise ... You need to have an internal capability to manage your APIs in a way that serves the business.

In this respect, these emerging movements connect up with recent trends in mainstream enterprise computing, where there's a move away from traditional integration middleware to a more flexible API-centric model. I was reminded of conversations with integration vendor MuleSoft's founder Ross Mason a couple of years ago about what he called the trend to API-fy enterprise IT. The goal — as ever — is a composable architecture, he explained:

[E]verything should be an API, and what we're really doing is stitching APIs together to perform different activities.

This composability — the ability to compose and recompose something based on any kind of new input or new application that sits on top — should be like the Holy Grail for many enterprises.

My take - from n-tier to tierless

Each of these movements have a different starting point, but they all seem to be converging on the same composable architecture for their destination. Established enterprise IT teams have further to go than others because of the need to 'APIfy' an existing infrastructure that wasn't built to participate in this new world. Cloud-native SaaS vendors and digital-native businesses are much closer. Last year I wrote about the emergence of a new tribe of enterprise application vendors, including Slack and Zendesk, who have grown up with a connection-first outlook that means they design, build and deliver their applications very differently than older incumbents:

They build on whatever technology comes to hand — open source and cloud infrastructure, connected services. For them, competitive advantage doesn't come from owning the stack, it comes from being free to select the best available resources for the moment ...

The conventional wisdom is to maximize what you own, but in today's hyperconnected cloud world, there's a new maxim — focus on whatever it is you can scale first, and faster, than anyone else. For everything else, use what's already out there.

That speed and agility is enabled by breaking away from the old pattern of building applications as a single, 3-tier stack of user interface, application code and database, or even the n-tier model of early composable architectures that allowed application servers to interconnect.

The new model is tierless — an open network ecosystem in which any function or resource becomes available through APIs to any qualified participant. Whether those functions and resources are data stores, microservices, system resources, serverless functions or SaaS applications and processes, the API layer makes them equally available as autonomous multipurpose composable services. They connect up to produce results and then present the outcomes to the end user through an engagement layer. This combination of headless engagement with serverless functions and resources defines the new architecture.

To work well, this new connected universe needs better tools and standards to ease connections and communications. As EPAM's Shekhter points out, it requires a degree of sophistication in creating and managing APIs, skills that are not yet mainstream — spreading this knowledge is one of the goals of the MACH Alliance. Data interoperability requires the evolution of common data models — early candidates include the Open Data Initiative led by Adobe, Microsoft and SAP, or the Cloud Information Model led by AWS, Salesforce and Genesys. There's also a long way to go to establish standard practices for managing large, constantly changing application ecosystems of the kind that the new breed of business systems professionals are grappling with. Sharing those practices in forums like today's virtual conference is the first step towards the development of tools and techniques that will emerge to ease this burden for others in the future.

Until all these various enablers fall into place, there will be plenty of scope for detractors to pour scorn on this new-fangled way of organizing enterprise IT. In the early days of any new approach, you have to be determined to choose this path, because the established way to do things is inevitably far easier and obvious. But there are enough believers out there to continue to push forward this new tierless architecture. It may yet take several years, but there's a clear path ahead for it to become the new mainstream.