Forget AWS Lambda, so long Kubernetes – this is the future of serverless

SUMMARY:

Forget about AWS Lambda and Kubernetes – AWS Fargate is the real future of serverless because it’s containers that matter, says New Relic’s Lee Atchison

New Relic serverless diagram 370pxLast year I wrote an article on what serverless computing is all about. In that article, I described that while serverless computing doesn’t remove servers, it moves the management of servers to the cloud computing provider, away from your development and IT organization. It removes complexity from application management and enables easier and more significant scaling by sharing server resources across a larger set of consumers.

But last year, when you said ‘serverless computing’, you were almost exclusively referring to Function-as-a-Service (FaaS) technologies such as AWS Lambda, Microsoft Azure Functions, and Google Cloud Functions. While there are other serverless technologies – such as serverless data stores and databases – these functional computing services were usually what you meant when you were referring to ‘serverless computing’.

AWS Lambda, the most prevalent FaaS, seemed to be a perfect solution. Write a function, deploy it, setup some triggers, and magically you had supposedly ‘infinite’ scaling. But AWS Lambda has some shortcomings. In particular, AWS Lambda has significant limitations placed on how the functions are executed. These include specifying which languages they can be written in and limiting how complex they are by significant memory usage and execution time constraints. These limitations focus the value of this service to specific classes of use cases.

AWS Lambda is good for small-scale compute tasks, but its use is limited as a generic compute offering.

Game changer – serverless done right

At re:Invent last fall, AWS introduced a new capability known as AWS Fargate. This is a technology that, when paired with a container orchestration service such as Amazon Elastic Container Service (ECS), removes the need for server management from standard container computing environments. In essence, AWS Fargate brings the value, convenience, and scalability of AWS Lambda to the container industry.

This service, in my opinion, is a significant game changer. AWS Fargate brings most of the value and convenience that AWS Lambda offers to a compute environment that is substantially less regulated and controlled. Containers can contain code written by anybody in any programming language and have nearly any amount of compute and memory requirement. Yet, with AWS Fargate, you can get the management ease and scalability advantages that have so far been limited to AWS Lambda. In other words … you get the best of both worlds.

Fargate is more than it appears to be

The AWS Fargate technology promises the ability to manage containers as independent, scalable entities of their own, without the concern of the underlying server infrastructure. This is similar to what Amazon EC2 offers to virtual servers – the ability to manage virtual servers without the concern of the underlying server virtualization layer.

AWS Fargate is to containers, what Amazon EC2 is to servers.

In my opinion, AWS Fargate is poised to offer easy, independent, scalable, compute without the need for understanding the underlying infrastructure. This is basically the best of Amazon EC2 and the best of AWS Lambda … serverless computing without the restrictions.

AWS Fargate will make containers easy to deploy and scale. Period.

Goodbye, Kubernetes

While this is a big promise on its own, I believe AWS Fargate can go one additional step. AWS Fargate can not only abstract out the servers the containers run on – hence the term serverless – it also can provide an abstraction for server orchestration, serving as orchestration-free computing for containers.

By leaving the orchestration work to AWS Fargate, you can focus exclusively on the thing that matters – containers. AWS Fargate can be both ‘serverless’ and ‘orchestrationless’.

This means that technologies such as Kubernetes will become as invisible to the consumer as server virtualization is today. Do you know how server virtualization works with your Amazon EC2 instances today? Is the virtualization something that you have to understand deeply to make Amazon EC2 function? No, of course not.

In the future, the same will apply to orchestration. You won’t need to know or care how services like AWS Fargate create and manage your containers … that will be the concern of the cloud service provider. You won’t know whether Kubernetes is involved in the picture or not. You won’t care about Kubernetes at all. It will all be managed by the cloud service provider.

Technologies such as Kubernetes will become irrelevant to us.

A misunderstood technology

AWS Fargate is a new technology, and its potential is not yet fully realized. We have a long ways to go before the real power and potential of AWS Fargate can be utilized. But I believe the future and power of AWS Fargate has been downplayed and misunderstood due to the hype cycle associated with AWS Lambda today. When all is said and done, I believe AWS Fargate-like services and serverless containers in general, whether from AWS or Microsoft or Google, will become a much bigger industry changing technology than AWS Lambda is believed to be today.

But only time will tell. What do you think?

Image credit - Administrator in suit adds cloud container to network © leowolfert - Fotolia.com

    1. clhodapp says:

      This article is nonsense. Lambda is more “serverless” than Fargate but the use-cases are very different: Lambda is for granular, short-running tasks. Fargate is for longer-running resources. It’s possible that in the future, Amazon will replatform Lambda to run atop Fargate, but that’s just an implementation detail that would reduce their maintainance burden. Kubernetes does do orchestration, but that’s not its true power. Instead, the real point of of Kubernetes is in the powerful abstractions that it gives for helping you operate your apps in a portable and highly automatable manner separate from the particular infrastructure that they happen to run on (e.g. let me put a load balancer in front of these containers… oh, it’s just an extra YAML file… Let’s mount a config file into my container… Oh that’s just a few extra lines in my Kubernetes config…). To that end, Fargate and Kubernetes are highly complementary: Fargate makes it easier to implement the Kubernetes model. Amazon is already working on a Fargate/Kubernetes mashup. And this isn’t even the only instance of this idea: see Microsoft’s work to do pretty much the same thing on Azure, under the title ‘virtual Kubelet’ (MS’s work seems to be a bit more technically-driven and narrowly scoped, so it doesn’t have a cool name and marketing angle like Fargate). All this is to say: Kubernetes and Lambda could be the “face” to Fargate, not the other way around. Anyway, I really feel like this shows a profoundly shallow understanding of the tech it is trying to cover… or else is just click-bait going for a salacious “X is the Y-killer” angle but… a reputable site like this would never run a story like that, right?!?

    2. Jeffrey Parker says:

      I don’t understand why Amazon had to release this before someone writes an article like this. Google had this same tech in their cloud for almost a year before Amazon. I also don’t consider this serverless as I believe that serverless would refer to you provide code and/or data and they take care of the environment that it runs on.

    3. Before writing off Kubernetes see @metaparticleio experiments by Brendan Burns to deliver “a compiler for Kubernetes”. Giving developers a way to spin up containers and deploy to Kubernetes in an app’s native code.

      After we saw JS client side developers transfer skills to NodeJS server side. I wouldn’t bet against developers doing the same here. Driving distributed cloud computing at scale by using the programing language they know throughout.

    4. Jane Feets says:

      Hey also Azure Container Instances came out before fargate but much more importantly ACI has a less opinionated architecture and they are truly abstracting away the vm layer with a connector into Kubernetes. Super exciting to see them follow suit though 🙂

    5. This article is more about tech monoculture rivalry (AWS, Azure, Google, IBM, Oracle) and the market landgrab underway in the serverless category–than about serverless technology itself.

      The rivalry between and among cloud giants is a constant in the tech industry. At Platformula1 we describe this as “landcape tectonics” and we assess this multi-dimensional rivalry continuously to provide actionable guidance to clients.

      AWS has proven to the world that it “eats” new partner-initiated categories they see being born inside their ecosystem with a “good enough” alternative. So Fargate is nothing new. It’s what AWS does on a regular basis. Embrace, extend, expunge. Monoculture economics to the max.

      But serverless is going to end up like SD-WAN. VERY fragmented market shared among multiple vendors. It likely wont play out the way the earlier cloud landgrab played out–because of the monoculture nature of big cloud.

      Many observers of the tech industry who only look at the developer issues without analyzing the accompanying issues of monoculture rivalry will miss the comng wave. “Goodbye Kubernetes” in the above post is just that–AWS-friendly marketing disguised as anti-Google tech analysis.

      The container wars are going to be around for a long time. It’s a big cloud cage fight.

    6. Victor says:

      do you mean AWS finally got a real innovation? wow big deal

    Leave a Reply

    Your email address will not be published. Required fields are marked *