Dropbox CTO on file sync and share at enterprise scale

Profile picture for user pwainewright By Phil Wainewright January 29, 2017
Summary:
In part one of this interview, Dropbox CTO Aditya Agarwal explains how the file sync and share cloud platform has been engineered for enterprise scale

Aditya Agarwal CTO Dropbox 370px
Aditya Agarwal, Dropbox

One of the strengths of file sync-and-share cloud provider Dropbox — but also an obstacle when the company wants to be taken seriously — is that people take its service for granted. They just use it, and it simply works.

Ease-of-use is a big part of its proposition to enterprise buyers, because rapid adoption is crucial to the success of any attempt to standardize on a single platform for online storage and collaboration. But equally, its simple effectiveness perpetuates the perception of Dropbox as a consumer-grade product, and belies the sophistication of the engineering at its heart.

Few, therefore, of its half-a-billion users are aware that last year the file sync and share platform completely migrated its entire core infrastructure off Amazon Web Services into custom-built datacenters of its own. CTO Aditya Agarwal, who I spoke to on a visit to London earlier this month, is still a little in awe of what his team achieved:

It still blows me away that we moved over 500 petabytes of data, which is probably the largest migration of data in history, and none of our users noticed. We had zero data loss. We had zero data downtime.

The fact that we pulled it off without any downtime, without any-user visible issues, is, in my opinion, a really big deal.

Because it just worked, the massive migration garnered fewer headlines than when, for example, an obscure bug fix temporarily resurrected some users' long-deleted files this month, or last year's revelation that user passwords stolen in 2012 have been released on the web. It's only when things go wrong that people notice.

Enterprise credibility

As CTO, a title he assumed in October after four years as VP of engineering, it's Agarwal's job to make sure things go wrong as little as possible. He's been the main architect of Dropbox for Business, charged with turning the sync-and-share platform into one that enterprises embrace. That's meant adding features such as Dropbox Infinite, which gives users local access to files without having to download the entire file store, and AdminX, a set of administration tools that provides the kind of granular control over access rights that enterprises demand.

As a result, Dropbox has already earned enterprise credibility among its 200,000 paying business users, including the likes of sportswear maker Adidas, which has 50,000 Dropbox users worldwide, media group News Corp, travel site Expedia and UK broadcaster Channel 4. Even so, there's more enterprise-grade functionality to come, says Agarwal.

We have a healthy number of large companies using us, but we still have a bunch of work to do around providing more admin control, more security, and so on. I think we need to go tackle that.

The introduction of Dropbox Infinite was a crucial step to move Dropbox away from its original focus on individual users. In the early days of Dropbox, the only way to share files across multiple users was for every user to keep a separate copy of the entire file store on their local device. That was unworkable for large companies, because if each user has their own 2GB store, by the time you have a thousand people that's already a 2TB shared store.

Infinite retains local syncing, but it only downloads files locally if the user's likely to need them. The rest of the files stay in the cloud in what is effectively "a company-level Dropbox," says Agarwal. It's a simple enough concept, but quite an engineering challenge:

It just has these massively difficult technical challenges. You're going from 2GB to potentially 20TB, in terms of the amount of data that your company has. How do you provide all of that data, only sync the data that the individual person in your company needs, hence Project Infinite, and how do you essentially make it perform and make it reliable and secure?

In order for us to get there, it requires some amazing technical innovation. The idea that you'll be able to go from having 2GB on your device to having potentially 20TB on your device, that's a pretty revolutionary idea.

One year from now, people will look back and essentially will not believe that the file syncing and sharing solutions out there do not work the way that Dropbox Infinite does. We're the only solution in the market that actually provides that. We're very proud of that.

Changing mindsets

The challenges are not merely technical. Taking this into the enterprise also requires some mindset changes, especially around security and data access, says Agarwal:

What you have really done is you have flipped the model around. In the 2GB world, everything is default private. Nobody can see, unless you explicitly choose to share it.

In that 20TB world, everything is default public, because you want everyone in your company to be able to access your company's data. You then restrict access as opposed to grant access. It's a pretty profound difference in terms of a privacy model. Once you have flipped the model, the security implications are enormous.

The most important contribution the vendor can make to help enterprises adjust to this new landscape is to give people tools that make transparent what they're doing, suggest Agarwal:

It's less about Dropbox and the technology but more about changing the culture in terms of how they think about access to data and security and privacy.

Our mental model, when we think about design and security for our products is that, the most usable products have the best security. If users are able to have a really crisp idea of whatever they're doing — in terms of sharing the data, viewing the data, accessing the data — then that is the most secure product.

Read on for more from Dropbox CTO Aditya Agarwal on advancing collaboration in the enterprise.