Solving eBay's elastic storage problem

Den Howlett Profile picture for user gonzodaddy March 6, 2014
eBay solved its elastic storage problem, when it turned to Nutanix, a startup in Silicon Valley that's disrupting the storage world with low cost solutions that take much of the administration away for those with large scale storage needs.

drew trieger
Drew Trieger - eBay

When most people think about storage they tend to think of it as an ever increasing set of electronic filing cabinets. That's not the case for eBay which has elastic storage needs.  This makes provisioning a significant challenge for a number of reasons. Drew Trieger, enterprise architect at eBay explained the problem like this:

We need to reduce physical capacity but we're not allowed to go rent elastic capacity from the likes of Amazon. There are security concerns on machines you don't control, there are budget concerns and latency problems. I imagine every company is going to move to infrastructure as a service and treat storage as a commodity but the problems I'm solving mean I want an elastic cloud in my data center under my own control.

That limits the options available to eBay and at one time, Trieger considered building his own version of an elastic storage network.

When we first looked at this problem, there didn't seem to be many options. We could have specified a solution and built it from scratch but then we would have to devote resource to managing and running storage when we could be putting that resource to much better use and I would not necessarily have ended up with a simplified data center.

eBay chose to go with a solution from Nutanix, which, at the time, was a new company in this space. Trieger needed some convincing that  this would be the right decision:

I understood their architecture was viable. It was sufficiently decoupled, share nothing and consistent. We wouldn't be  overusing one channel of the hardware and so creating strain and cost. I prefer to have infrastructure that all stretches together. They've got this thing well balanced between CPU, disk, storage and memory.  I want to scale dynamically and they said that was part of the solution on offer. The way the deal works  puts the responsibility on Nutanix to scale reliably. They can tune the hardware and software together to provide me with a consistently performant solution. When it comes to cost, they're only a 20-30% premium over the raw hardware so when taken all together, it was a relatively simple decision.

Several years on and eBay believes it has the right solution:

It saved us a good six figures. I don't need to have a storage network and don't need people to maintain it. At scale it could save us millions. All the setup time disappears and Nutanix handle all the support for us. We rarely tune the solution which is a massive bonus. When one thing gets to be a bottleneck then we can choose to lose some of the network for a short while. If I want, I have options to deploy to my internal cloud or other cloud. In other words, I've got the flexibility I want but with someone else handling the hard work.

If all this sounds like motherhood and apple pie, Trieger warns that Nutanix isn't the right solution in every circumstance. He doesn't for instance think that startups should consider this as part of an early infrastructure strategy because the upfront investment is significant - in the low six figures.

However, for those businesses with deeper pockets, Trieger says that Nutanix has delivered what they promised and are doing a lot of pro-active work in for example the remote administration of multiple systems. That talks directly to scale, a crucial issue at a time when data volumes are increasing at a staggering pace.

Featured image: Flickr/liewcf


A grey colored placeholder image