Why enterprises need a distinction between multiple clouds and multi-cloud - the latter requires careful network planning

Profile picture for user kmarko By Kurt Marko February 17, 2020
Oracle’s announcement of the expansion of its inter-networking partnership with Microsoft to Europe reflects the necessity of such services as enterprises opportunistically employ multiple clouds for different features to build true multi-cloud composite apps.


As commonly used, the term 'multi-cloud' doesn’t imply anything about a user’s application architecture. However, once organizations with more than one set of cloud services start using them cooperatively and synergistically, they find that the design and implementation implications are far more significant than checking a “multi-cloud” survey box.

That’s because, carefully crafted web-scale systems that anticipate a highly distributed and occasionally hostile network environment, most enterprise software is designed with the assumption that interdependent systems can easily communicate with each other with little resistance, i.e. latency, jitter, packet loss and bandwidth throttling in network-speak.

Once an application is spread across multiple clouds, these assumptions go out the window if the services are interconnected, as is the case for most nascent cloud deployments, over the Internet using VPNs. In these cases, users are in for a nasty surprise if they expect the same repeatable performance they got from on-premises systems.

Instead, IT organizations with distributed multi-cloud systems must pay as much attention to the network stitching clouds together as they do to the on-cloud application design and service selection. Unfortunately, it’s not in the best interest of cloud providers to make it easy to move workloads and data back and forth to competitors. Indeed, most charge transfer fees for the privilege.

That’s why the Oracle-Microsoft cloud interoperability partnership announced last year and recently expanded to Europe is such a significant deal. Sure, similar capabilities are available from carriers, like AT&T NetBond and CenturyLink Cloud Connect, or colocation providers like Equinix Cloud Exchange and Switch Cloud, but why employ a separate connectivity broker when you can directly link two cloud providers using a native service? Indeed, according to a report from Accenture, setting up such telco interconnections “can cost as much as $100,000 per year in recurring charges,” on top of setup fees.

Let’s go into more detail on the problem and why I hope the Oracle-Microsoft detente leads other cloud providers to break down the walls separating them from their competitors.

Multi-cloud adoption isn’t all it’s cracked up to be. 

Most software vendors now pay homage to the cloud as an alternative platform for their products and have seen the advantages of cloud infrastructure. SImilarly, many IT organizations figure that if one cloud is good, two or more must be better. Consequently, read any survey measuring enterprise cloud usage and you’re bound to see a growing majority of respondents claiming to have some form of a multi-cloud strategy.

Similar to the ambiguity around the term “hybrid cloud,” “multi-cloud” has been the subject of much imprecision and wishful thinking. For example, when we refer to a hybrid car, we mean a vehicle with two modes of propulsion — usually a gas-burning internal combustion engine and a battery-fed motor — that work in concert to provide the required amount of power for a particular situation. Deductively, we would conclude that a hybrid cloud architecture would be two disparate clouds — generally accepted as an on-premises virtualized software stack and one public infrastructure cloud (IaaS) — used together to provide application services and resources. Unfortunately, that’s not how most people use the term. Instead, hybrid usually means connecting on-premises and public clouds where workloads and data occasionally move between the two; for example, replicating on-premises applications to the cloud for disaster recovery.

The same semantic sloppiness besets the term “multi-cloud”, which surveys typically don’t define. This results in responses that reflect the lowest common denominator definition, namely an organization that uses one or more cloud services. Such a broad definition eviscerates the term of meaning since it implies that the three-person accounting shop using QuickBooks, Dropbox, Office 365 and CloudBerry is just as much multi-cloud as the multinational corporation using AWS, Azure and Salesforce.

We need another term for such cloud promiscuity that doesn’t conflate cloud usage with its more sophisticated cousin, a cooperative, integrated infrastructure that can provide an application cloud services from multiple providers. A decent first attempt is “multiple clouds” for the former versus “multi-cloud” to describe the latter, although this suffers from the fact that these two terms have dictionary definitions. “Hybrid multi-cloud” is too cumbersome and beset by hybrid’s existing misuse, but until seeing a better alternative, I’ll stick with the multi-multiple distinction.

The multi-multiple semantics might seem unimportant, but it makes a significant difference when looking at the network infrastructure required to stitch clouds together. As I’ve written twice (here and here) over the past 18 months, the network performance of the three major cloud providers, AWS, Microsoft Azure and Google Cloud, can significantly vary depending on many factors, including:

  • The cloud provider, since each uses quite different network designs and topologies.
  • Region pairs for intra-cloud links.
  • Location of client accessing cloud services and the cloud region.
  • The ISP used for cloud access.

Add these performance variables for traffic within a provider to those when bridging to clouds, say AWS and Azure, and you have a recipe for network unpredictability. The surest way to solve these issues is by building direct links between clouds that are operated by the providers. Given their competitive rivalries, such cooperation seems about as likely as Apple and Microsoft agreeing to let each other’s operating systems run on their competitor’s hardware, but that’s precisely what Microsoft and Oracle have done.

Oracle-Microsoft interconnect lets customers use the best of both

Oracle isn’t a cloud market heavyweight, however, Oracle Cloud (OCI) is a compelling option for organizations with complicated Oracle database applications that wish to migrate them to cloud infrastructure. Perhaps it’s because of Oracle’s minuscule market position that Microsoft chose to work with them. Regardless of the motivation, the two have partnered to create a peered network that links the two clouds so tightly that composite applications might not know which is supplying a particular service.

The Oracle-Microsoft service links their direct private network services, FastConnect and ExpressRoute respectively, which provide fast, secure links between a customer location and the cloud service. While these private circuits typically provide isolation from all other Internet traffic, the interconnect allows the two clouds to talk with each other. Such cloud-cloud communication is extremely advantageous when a legacy Oracle databases and associated code has been migrated to OCI’s autonomous DB service, while new enterprise applications that need to access these systems are built on Azure.

However, as the diagram below illustrates, the link isn’t transitive. A customer can use ExpressRoute to access Azure and FastConnect to reach Oracle cloud, but can’t pass through one to the other, say, use an ExpressRoute connection to access Oracle via Azure. The link is only for service-to-service communication.

kurt image

The interconnect itself is created by using each service’s virtual network and gateway products, which enable traffic control and security policies to restrict the types of traffic and source/destination pairs allowed to use the circuit. Since each service (FastConnect and ExpressRoute) supports up to 10gbs circuits and the ability to bond links together, it allows organizations to bind the two clouds with an blazingly fast connections. Indeed, Accenture’s testing found that the average latency between clouds was less than 1.5ms. For comparison, the ThousandEyes report I cited in my earlier articles on cloud network variability found that the latency between Azure Availability Zones is about 0.8ms, while that between users and Azure regions in the US runs about 40ms.


Initially, the Oracle-Microsoft service was only available in the U.S. East region (Ashburn, VA), however this month, Oracle announced its expansion to Toronto, London and Amsterdam.

The MESTEC example

Using two clouds to exploit the best features of each will become increasingly popular as enterprises transform IT to higher-value activities and eliminate operational overhead by shedding data centers in favor of cloud services. MESTEC, a UK-based provider of manufacturing execution systems (MES), is one such company recently profiled in diginomica that uses OCI for databases and Azure for Web apps. Derek du Preez's interview with MESTEC CTO Mark Carleton detailed its multi-cloud use case and concerns (emphasis added):

Carleton added that MESTEC’s primary concern about operating in a multi-cloud environment was performance - the company has always had its application and database in the same data centre. He said that this concern turned out to be unwarranted.

[ Quoting Carleton ] If we split those things apart, what’s that going to do for latency and performance? To our relief, and our surprise, we found that it behaved better over public Internet than our legacy infrastructure had in a single data centre. The benefits of the best of breed PaaS in those two environments far outweighs the disadvantages of having those things on different clouds.

Carleton noted that some workloads run five-time faster on the split-cloud architecture using the new interconnect than they do a private data center. He is also pleased with the autoscaling feature of Oracle’s database service and the fact that the interconnect is directly supported by Microsoft and Oracle, which eliminates the all-too-common finger-pointing when networking issues arise between two vendors.

My take

As enterprise adoption of cloud services evolved from one-off installations by test and development groups to being part of its core IT strategy, the need for more robust cloud networking became apparent. During this transition, organizations with growing cloud footprints typically saw the limitations of baseline interconnects using VPNs over the public Internet and sought high-performance, predictable and secure alternatives. Cloud vendors responded with private circuits such as AWS Direct Connect, Azure ExpressRoute and Oracle FastConnect.

As enterprise cloud maturity increases to the point where enterprises synergistically use multiple clouds, cherry pick the best features from each and combine them into composite applications. At that point, organizations will face the realization that inter-cloud networking needs a speed-boost via private backend networks between providers. In this environment, services like the Oracle-Microsoft interconnect, along with other features of the partnership like federated single sign-on and cooperating support organizations, will become requirements for the remaining cloud providers.

It’s time for the major cloud vendors to lower their lock-in ambitions and embrace the multi-cloud needs of more and more enterprises. Universal network peering between clouds and supported private link services won’t just allow organizations to build composite applications using best-of-breed cloud services, but it will increase cloud competition. The result will be better products as cloud vendors improve their offerings to retain customers based on the merits of their services, not the friction of moving to a new supplier.