Putting the new Jenkins to work at HSBC

Profile picture for user mbanks By Martin Banks December 11, 2018
How the bank’s Global Head of Digital Platform Operations is transforming applications management and deployment by using the latest version of Jenkins to start the move off-premise.

Banking may once have been a rather staid, fuddy-duddy business, but the combination of IT and globalised competition means that the faster a bank can react to new market demands and competitive actions, the better it can be at maintaining a hold on, or ideally growing, its market share. And when that bank is a major global player, such as HSBC, the stakes are obviously high.

It was against this background that Cheryl Razzell joined the company as Global Head of Digital Platform Operations some two years ago. The bank was a long-time user of Cloudbees’ Jenkins automated application lifecycle management tools, but when she joined some problems were starting to show up in the existing on-premise Jenkins-based infrastructure. Like many other users, the bank was aware that the on-premise version of Jenkins was getting close to hitting something of a wall and was starting to run out of legs.

As Cloudbees has indicated, it was well aware of this problem, and Razzell’s observations show that the company was sailing close to the edge:

I think the model needed to change, to adapt to the market demand. I think our Internal development teams would have superceded what Jenkins was able to offer if Cloudbees hadn't adapted as quickly as it did.

Applications development teams within the bank were starting to think in terms of developing in-house solutions to meet what was now required. But as Razzell observed, the company listened to what customers were saying, and especially about containerisation and the cloud, and the bank is now working to build out new applications and services, as well as bring forward existing on-premise applications into the cloud, using the latest versions of Jenkins.

Sharing out the autonomy

This is particularly important, for many of these existing applications are in use by the retail operations of HSBC, and this is now where competition for customers is at its most obvious and aggressive. The speed with which her team can upgrade and extend applications and get them into production is, therefore, a key requirement.

The project is still at a fairly early stage, with development work establishing a solid footing in the new in the new world of cloud and Jenkins:

We have to do some performance testing to make sure that it meets our requirements. So there's a lot of pre-work before we validate whether or not we're going to move over and start using it in production. So if we bring something up as proof of concept, we're still running an older version of Jenkins.

But this is now delivered from an instance running on AWS, and the bank still has teams deploying applications on that environment. Some of these applications are, however, due to go live in the near future.

At the same time, her team are also running a third development project based on using Kubernetes with Jenkins. This is being tested as a migration path to a new Jenkins AWS, which will be what the bank has running on premise, but in a highly scalable, automated solution. In infrastructure terms this is the same version of Jenkins, but with the advantages of running in the cloud.

This will have segregated development and production environments and will allow teams to scale both up and down on demand. It will also allow teams to build and spin up their own instances with individual dashboards so they can monitor and look after their own environments. This will give them a lot more autonomy to support themselves.

Got to start somewhere

This is when Razzell expects to see the start of development of new applications that exploit the cloud. The aim is to provide the business group development teams with a stable and consistent Continuous Integration/Continuous Delivery (CI/CD) platform that provides the environment for new applications development while maintaining consistency with existing tools:

The idea of the digital transformation project is that it has to start somewhere, we can’t transform the whole bank in one go. So it starts in retail, and other teams are also starting to look at the way we are using CI/CD and how they can move off some of the older legacy platforms and move to the new digital platform. So we're working at the moment with our Jenkins in AWS to get that set up so that our teams can start deploying on that and then we'll start to work at another parallel stream to bring in the Kubernetes and Dockerisation of our Jenkins environment.

Though her team are currently helping divisional development teams start on this journey the goal is to make them all self-supporting. The one thing she does want to avoid is creating a different, more modern version of the classic `Jenkinstein Monster’, the centralised complex Jenkins environment that has tended to grow around the group with the best Jenkins skills in most on-premise installations – here the common cry from divisions and departments is: `you understand this stuff, can you do it for us?’

The goal, then, is to be able to build an automated, unsupported cloud environment that provides development teams and users with a zero-touch infrastructure. Part of this will then be an applications estate that is being regularly rebuilt. The goal at the moment is a rebuild every 21 days around a three tier model similar to the ancient 'grandfather, father, son'  file backup approach. Here, 'grandfather' is the current production version, 'father' is the next version nearly ready to go and 'son' is the start point of the next early-development version. This is where new developments can be tested:

That way there's no disruption to the service. The teams will carry on using their infrastructure, but will have the latest version of the operating system that complies with AWS, Google, or whichever vendor we’re hosted on. That makes sure that the infrastructure and the applications are kept up to date.

Following the announcement of some major Jenkins upgrades from Cloudbees the company has, in Razzell’s view, moved significantly from drifting behind HSBC’s needs to being well ahead of them, which puts her in the position of looking at the possibilities that are now available to her team. But she is also well aware that, as transformation moves forward, and as both development teams and end users get more experience of what is on offer, that situation may now change rapidly:

I think the roadmap is pretty feature-rich in comparison to our needs, but I don't know what we might need next. And things crop up, technologies change. You could ask me next week, and maybe I would feel completely different, so until you start using something, you're never going to know what you're going to need from it.