However, Travelers Business Insurance is taking steps to improve the speed of delivery, reducing the time it takes from a business ask to enter production, through the introduction of agile processes, microservices and NoSQL database provider, MongoDB.
I got the chance to speak to Jeff Needham, senior architect, and Michael Braasch, vice president, of Travelers Insurance, about the company’s challenges and plans for the future. By way of background, as an insight into the set up of the company, Braasch explained:
Within Business Insurance, we were tasked with building out an application to support our underwriting activities. We really set out to modernise our application, just within business insurance. Previous to our application, our users would have to log in to up to a dozen different applications just to manage their book of business. And so we had a bunch of legacy systems, obviously, that we had to deal with and integrate with. In the course of building out our applications it became obvious that we were starting to build this monolithic web of services, to a single database. That was starting to become our choke point.
So, we said that now it’s time to look at something different. What’s the next disruption and what’s at the database level to get us to that transformation?
Enter MongoDB. Needham explained that in the lead up to this new application, everything was aimed at reducing the time form business ask to production. As a big company, with lots of systems, the integration challenges increase over time. And as Braasch noted, the database was proving to be the sticking point. Needham said:
If you’re sitting around waiting for three days as a developer for a database change to be implemented, in an eight week release, that’s a fair amount of time. That’s a quantifiable amount of time that you’re losing. Everything that we’ve done, from adopting the scale agile framework, to moving to microservices, to moving to continuous delivery, is to make sure that we are getting faster with what we do. But at the very end was still the database. We implemented all these things, but we still had to wait two or three days for a database change to be implemented. It didn’t really add up.
Needham said that Travelers Insurance was frustrated with Oracle and the relational model, and that MongoDB was the breakaway leader in the NoSQL space, as the JSON schema was human readable. But how did the company end up with MongoDB? The change occurred when Travelers realised that it needed to shift to a model whereby single services were responsible for their data changes. Needham explained:
We have almost 150 microservices. And we built all these application functionalities on top of a single database. And it really didn’t take long before the accounts screen and the policy screen, were all kind of sharing the same table under the covers. So if we made a schema change to the table, not only do I break your service, but I break the other two that are consuming that data.
So we went to microservices and said, no one is going to touch this table, but this service. So if you need that data, you can call that service. So I can break schema, or change schema, and the only thing I have to manage is that one service. That was great. That was really the first evolution towards MongoDB, which just seems so logical. If we are letting a single team own their database change, why not let the team make the database changes themselves?
But you can’t do that with Oracle or SQL Server, you still have to have this DBA skill set to manage schema and manage database changes. So, for us, looking at the options, documentation and features, MongoDB was right in line with continuous delivery. I can make database changes as a developer, I don’t have to hand that off to another team. The one microservice that owns the table now, that can be managed by the one team that owns the database changes now. They’re a self-contained unit.
Benefits and challenges
There have been a number of benefits realised as a result of this shift towards continuous delivery and the use of MongoDB. The first and most obvious one being, that there has been a reduction in time and greater speed has been introduced. Braasch said:
It’s really about speed. By limiting that work in progress, we are not waiting on hand offs, we are not waiting on other areas. In the past, it’s been more project based. Long, big projects, deploy at the end of X number of months. Now we are able to focus on things in smaller batch sizes, so build something, move it into production and get that real-time feedback from users.
However, the other main benefit, as a result of MongoDB’s schema, is that Travelers has been able to bring all teams along on the journey, not just developers. Needham said:
Developers love this database. But now I’ve got a team of analysts, I’ve got product owners. Those people play a role in your delivery. And I think if they become alienated - like, if you were to choose Hadoop - I can guarantee you that those folks would have no idea what you were doing. They wouldn’t be able to see and touch the data and understand it. They would feel alienated and not part of that process.
I think that with adoption, you need all those folks to come along on the ride with you. And I think MongoDB’s JSON schema, it’s human readable, it’s not complicated syntax. We taught all our BA and QA folks, here’s how you model the information in JSON, here’s how you query the database, here’s how you write data. Just so that you can touch it, feel it, see it, so that’s demystified for them.
That made a really big deal, because they were excited and part of this effort. It wasn’t this cryptic data format that only the developers could understand.
However, this also plays into the greatest challenge that Travelers has experienced - the people and cultural change. Braasch said that it has been a huge cultural shift and that for an organisation to get to a point of continuous delivery, they can’t just focus on the technology. He said:
What are the processes around it and the skill sets of the people? That’s certainly been one of the challenges. Adopting new technology, that’s not usually the hard part. We’ve got a lot of great engineers that are always interested in looking at new technology. In many ways, that’s the easy part. How does the skill sets of the people need to change? That’s been something that’s been pretty different for us, because we are a pretty large organisation.
It’s probably been slower than we would like. At times there’s frustrations around that, but I think that when we look at new things, our goal is always start small and run some experiments. Even with Mongo, we did it for one team on one idea, one concept. So even though we had our entire application, we didn’t have this big transition to Mongo roadmaps.
Are we still using relational databases? Yes, we are. We will make this shift as and when it makes sense to. Not everyone marching towards that. That’s helped us to enable this change. Starting small. We want it to be contained and we want to be able to make pivots.