Defining itself as an organization whose purpose is 'taking care of one of life's essentials', Severn Trent, one of Britain's largest water companies, delivers water services to 4.6 million people across the Midlands-from the outskirts of Sheffield, down to Bristol, and into north and mid-Wales.
That essential service includes delivery on a daily basis of 2 billion litres of potable water, and the carrying away of 3.2 billion litres of wastewater. Doing so clearly involves both extensive physical infrastructure, but also extensive IT assets too, with an internal IT leadership structure devolved into delivery, security and development.
A key pillar is what the firm has designated as ‘Tech Ops', defined as maintaining and improving the firm's live environments and assets and which is overseen by a long-time veteran at the company, Mark Gwynne, IT Director for Service Management.
Recently, Gwynne has had to solve a sticky problem: exit a physical site as part of a structured, enterprise-wide move to Microsoft Azure. So far, so familiar, perhaps: this is surely the archetypal move to cloud many CIOs have negotiated these past few years?
Retaining some on-prem
Well, not quite - as due to the regulated nature of the industry his company works in, several core applications could not be moved into the cloud, yet the clock was still ticking on that building. As Gwynne explained:
We've been on our cloud journey over the last 18 months, moving services from our data centre into the cloud, and we've got to about 65% to 70% of the footprint into Azure. But like most big programmes of this nature, we still had some older technology and there were some regulatory aspects where we needed to retain some of that infrastructure on-premise.
We had a building in a prime location in Solihull for over 35 years, which we had built up as a data centre; at one time, we had almost 180 racks in a data hall in there, but the lease was about to come to an end, and we had a new strategy of moving to being cloud-first organisation, and this was where this retained element resided.
We didn't want to continue to invest in any kind of traditional on-premise infrastructure, and we also saw a significant potential saving that could be made from a business perspective from not having to pay rent on the asset any more. You're obviously talking about a big power bill, too, plus business rates, that we didn't want to pay any more, too.
The problem was the applications, which were both needed to be retained (still run on physical hardware in an identified UK location) but which also happened to be rather mission-critical. Gwynn said:
This was both a significant chunk of IT and significant business systems for the organisation; a lot of core applications like SAP modules had already been moved into Azure, but we still had left our main customer billing systems and our telemetry platform that looks after all of our operational assets.
These really matter to Severn Trent, as without telemetry, we're blind to what's happening across all of our operational business, and without billing, obviously we can't service our customers.
No cutting corners
Gwynne solved his problem by working with Sungard Availability Services. He explained:
We've been a customer for probably eight years or so, and they had been providing a site in London for us as our primary data center, so we had a good partnership with them already. We needed a second site for all of our disaster recovery and all of our backup to the primary for what we were retaining, so they were kind of the obvious choice for us to do that, given our existing good relationship.
Specifically, the Solihull data center was shut down on time as planned and these key business systems have been ported via dual-site colocation for both production and recovery environments at the latter's London-based UK data center for recovering all our IT services, as well as a workplace recovery facility in Coventry with the vendor.
This was done by a close partnership with Gwynne and his team via a range of tools, including consulting on the new data center's design, a raft of specialized data center migration services for secure equipment transportation and overall project management, he confirmed.
That all sounds well and good - but as anyone who's ever been involved in a migration like this probably knows all too well, it can be just a little hairier in real life. Especially in a pandemic, of course. Gwynne added:
We had to be out of the data center by January 2021 because of the terms of the lease with the existing landlord. We'd committed to be out, but obviously COVID-19 hit right in the middle of the programme, so we did some rearrangement of our plans in terms of migration of services to the cloud and the order in which we did those. But we went from, obviously, working as a big programme team in the office to working remotely on videoconferencing, collaborating in that sense but we still needed to achieve the target timescale, even with COVID.
In the end, we did a lift and shift over a weekend of moving a lot of the main infrastructure down to London from Birmingham in December 2020.
Good planning was important in making that critical weekend go off right, Gwynne said:
We had to do it right. We couldn't cut any corners at all, because the last thing we can afford is for any of those critical services to not be available for an extended period. And as you can imagine, you don't really know necessarily how well that is going to go on the day-and given that you've got 35 years of history in that data center, as you move things or as you switch something off if it's going to have some particular impact on the business.
We did have a number of impacts on the day that were unforeseen which we had to deal with and change the timeline a little bit, but the vendor was really flexible around how it worked and gave us the time and opportunity to still get all of the equipment moved down to London. In fact, its team was working all through the night, all through early hours in the morning through the entire weekend, to make sure that we were good to operate as a business from the Monday morning.
The results speak for themselves. Severn Trent reckons to be on track to make at least £1 million a year operational savings from exiting the costs of the Solihull operation, plus additional IT savings from only paying for "the racks we use, not the building we have''. But that's not all, as Gwynne explained:
There's definitely a security benefit as well; we are inherently more secure now versus trying to manage all of our disaster recovery and security in our own data center. I also think there's a flexibility aspect here, as we're able to scale either up or shrink down in terms of our footprint, whereas before it was sunk cost at our own facility.
For Gwynne the most surprising thing about his experience here is how well people come together as a team in tense situations, he said:
To manage such a critical move for a critical service, everybody pulled together as a team and there was so much positivity.
We have some critical values as a company that include taking pride in our work, showing care and having courage-and I think all of those shone through in this project.