Ordnance Survey and the nine-second server

Ordnance Survey has moved to all-flash storage to support its extensive use of virtualization, delivering improved stability and reduced performance issues with its main database.

Due to the move, the national geo-location organization, which provides location data to public and private sector users, can also build new servers much more rapidly than was possible using older, physical systems.  

Since 1791, Ordnance Survey (OS) has mapped the nation by creating, maintaining and distributing detailed location information about Great Britain. The OS is responsible for keeping 500 million geospatial features up to date, in a central 4-terabyte master map.

Now, the fully government-owned company is seen as a world-leading, reliable geographic framework that delivers effective, efficient services to clients. These include UK emergency services, utility providers, transport networks and the Land Registry. The digital side of its business now accounts for more than 90% of turnover: OS is also helping developing countries to map their landscapes, including Zambia and Guyana.

20,000 database changes a day

That main OS master map, the Geospatial Data Management System (GDMS), is one of the world's largest spatial databases. It records every geographic feature in the country down to individual roads, buildings and parks. At one time, the database was the world's largest Oracle production system. But a decision taken several years ago to start exploring the potential of virtual infrastructure meant that an on-premise solution was no longer viable.

Moving to virtual technology required a storage solution that would offer very fast access to data, said Lloyd Clarke, Virtualization Lead at the organization. The ability to perform snapshots and deduplication was also highly desirable. Why: snapshots of the large OS database are crucial for back-ups and disaster recovery, while deduplication saves storage costs and reduces access time.

Finally, as GDMS may need to be updated 20,000 times a day, any new system would have to be able to support that level of change in a timely, safe fashion. Clarke said:

The conventional storage with spinning discs we were using just wasn't fast enough. It takes a lot to maintain a database with half a billion features-there's a lot of transfer in and out, for example, so we were always adding discs. Not only were we taking information out, but we also have surveyors on the ground who need to be able to make changes, too. We needed to write those changes back, and a lot of these changes are continual and around the clock.

Finally, rather than having separate databases, we wanted to have a single geospatial database that we could extract what we want from, rather than maintain multiple databases.

Performance could also be very spotty, Clarke added. This was particularly true at peak times in business hours, when traffic spikes. The chosen solution-implemented in 2010-is an all-flash storage approach using products from Pure Storage called FlashArray.

Clarke said:

We went out to tender, and this was the one that came back with all the ticks, from access to compression and levels of deduplication that we weren't even expecting.

It's important to note that this technology operates in a hybrid on-prem and cloud environment. More than a decade later, the organization has not completely moved off physical hardware:

OS has a cloud-first policy. But we tend to have more of a cloud-appropriate policy, where we look at whether it fits in the cloud, or whether it's still better to be on site. We have a phenomenal data centre here in Southampton, after all, and sometimes it just makes sense that hardware and the storage is located on-site, rather than going up to the cloud. But we do have a very large, virtualized estate, currently 1,700 virtual machines.

‘Within seconds'

The main benefit of the all-flash storage has turned out to be the expected ability to handle constant GDMS edits and additions. These can then be quickly integrated into OS mapping products.

The system is also extremely reliable and well-supported: Clarke stated that the vendor often alerts him to an issue before users, delivering more or less 100% uptime. But for Clarke, the main standout has been the ability to deploy servers and spin up virtual machines in minutes. This allows Ordnance Survey to react to customer needs far more quickly than with the older technology, he said:

In the old days, when you put in a request, you might need to have provisioning of storage. That meant buying new discs and it could be weeks before you had your service up and running. Now, it's just done on the fly. You can whip up a server in pretty much no time at all, which means no slowdown in work.

Internally, this rapid resource creation is known as ‘the nine second server' creation process, he added:

Sometimes it's a few seconds faster, a few seconds slower, than nine seconds. But the point is that it's really, really fast compared to what we had before. You give the system the spec of what you want as a virtual machine, click a button - and within seconds, you have a virtual service up and running. That's also with the operating system all configured and ready to go. Before flash, we couldn't do that.

Ordnance Survey anticipates no major changes to this form of storage. That is because it both continues to deliver against organizational need, and allows the team to concentrate on innovation:

This does its job, which of course gives us time to do more ourselves. We are a technology company, so to be able to spend more time on that rather than worrying about discs can only be a benefit.

Finally, summing up the experience of moving off ‘tin' and adopting new flash and virtualized ways of working, Clarke said:

We no longer have racks and racks of storage, so we have massively reduced our carbon footprint. We can also provide critical, up-to-date and accurate data to our customers that we couldn't have done without this technology.