OpenStack Summit – Bloomberg urges developers to engineer more, operate less
- Summary:
- For those businesses starting out on the job of transitioning their IT to a cloud-based environment, the news and business information provider advises that a Tapas approach, rather than an all-in, 4-burger pig-out, is the way to digest the job.
Take news and business information provider, Bloomberg, for example, which has appeared in the breakouts in the past to talk about its plans and aspirations. But this year Jacob Rosenberg, of the company’s infrastructure team, had a different agenda. He came to confront an accusation that has often been made to him by other users, namely if OpenStack is so great, why not run everything on it?
He made it clear that it was not beyond the bounds of reason that this might be the case at some point in the future, especially as Bloomberg had gone from the proof of concept stage to point where it is starting to put cloud applications into production. But there is a stage to be gone through first, which he defined as the point where people realise that reality is rarely as good as the dream. There will always be real life problems in any move from one environment to another, be that extensive technical debt, or adaptations, modifications and patches unique to a business.
So while the fantasy of 'drag and drop' porting of existing business applications may fade rapidly, his advice to the audience was that it should not be a time for despair. Instead, it is a time when IT teams should be ready to face up to their demons, which is what Bloomberg has set about doing.
His fantasy state is to have everything built on open source software. The company does use it, and it contributes to projects. The fantasy would also include the widespread use of commodity, `open’ hardware as put forward by the open compute project. He would also like to see horizontally scaled applications, for they tend to avoid single points of failure.
Lastly, he would also like applications to use cloud native design:
But the reality is that technical data accumulates and we do not always get the time to clean it up. And that can be huge amount of work, especially if it involves refactoring an existing application to be cloud-native when it has a couple of million lines of code.
The reality of reaching that goal of running everything on not just OpenStack, but any new platform, is that job of wading through the work of refactoring existing applications so that they can take their place in a cloud-native environment, and he outlined the steps that Bloomberg has gone through – and is still going through – in order to reach that goal.
The key thing to avoid, he suggested, is trying to do it all at once, in one lump. This is a time to embrace incrementalism and, while it may not be possible to go in the right direction straight away, try and avoid going in the wrong direction:
That process includes sharing the vision of what needs to happen with others so they can contribute thought and opinion.
Many legacy applications can be fenced off behind APIs, using such tools to create their interfaces to the new, cloud native applications and business processes. This is particularly important when dealing with legacy applications that still have a major role to play in the overall business process, says Rosenberg:
Incrementalism is all about building scaffolding, and the thing about scaffolding is that it is taken away before the building is finished and opened.
He also suggested that developers should learn to do what they can to refactor an application, when they get the opportunity to do so. This means taking the opportunity to fix little bits of the application, as they come across a problem, every time they are inside the code. His view is that there is always some small bit that can be fixed now even if the overall plan does not yet call for an attack on a major redesign stage.
Engineer more
Rosenberg also advised developers to engineer more and operate less. In other words, there is a need to make sure that what actions are undertaken do actually go somewhere constructive:
For example, consider automating everything you can, when you can, as the transition to the cloud is made. Another factor is to modernise the packaging – so that means using containers, obviously, but other things are possible. Finally, be ruthless.
His target here is the need to be quite tough on deciding what applications make it through to the cloud and what can be left at least 'as is’, if not specifically retired as end-of life:
Not everything will make it to the cloud – especially those applications that have tough or awkward licencing requirements, or need special hardware. So leave them to last, or see if the process can be re-engineered as a new application. But don’t get into work like that if it seems it will become a real heavy lifting job. But don’t be afraid to do other jobs first.
There will be a need to tear down organisational walls, and this applies to infrastructure as much as anything else. Most importantly, everyone in the business will have to learn how to use the new cloud applications and services, so plenty of training will be an essential part of the mix. This means being willing to share all relevant information such as metrics, logs and operational data, concluded Rosenberg:
Remember that the cloud is a fuzzy subject, so sharing helps. There is a need to provide engineering self-service and a need to trust people to do the right thing. This will also help them to learn and how to work together to get things done. There will be less demarcation between who does what with the cloud, so there will be a need to help them to help themselves.
My take
Some good advice on the ground work required for any business making the transition from an on-premise infrastructure to the cloud. Like many similar jobs, it is not easy or any kind of cakewalk, but it need not be as scary as it looks at the start.