Transport for Greater Manchester is using Snowflake to create a real-time boost to journey times

Mark Samuels Profile picture for user Mark Samuels March 10, 2020
Head of IT Malcom Lowe explains how cloud-based data warehousing vendor Snowflake is helping to produce insight on transport issues and sponsor timely interventions.

Image of Manchester City

Transport for Greater Manchester (TfGM) is using Snowflake’s cloud-built data platform to create insight-led interventions that help to improve network efficiencies and boost passenger experiences.

TfGM is the local government body responsible for delivering Greater Manchester’s transport strategy. More than 5.6 million journeys are made across the region’s transport network each day. TfGM works with bus, tram and train operators and oversees some of Manchester’s busiest roads. The organisation implemented Snowflake in July 2019 to help it generate insight from the huge volume of data that’s associated to passenger journeys.

A key factor in this decision was the simultaneous opening of the Metrolink network in July last year, which is an intelligent contactless transport system and the largest light-rail network in the UK. Malcolm Lowe, head of IT at TfGM, says the organisation was keen to make the most of the growth of information associated to this new transport system:

We knew we were going to get a lot of rich and useful data to understand the impact of contactless ticketing, but we knew this information could also help us improve other transport services as well. So, for example, we could find out where the bottlenecks were and we could change pricing to see whether that changes travel patterns. As an organisation, we try not to think in silos – we try and think organisation-wide. We needed something that supported contactless ticketing, but which could also help support the rest of the organisation.

TfGM proved the potential benefits of a data-led approach by using a proof-of-concept based on Azure SQL Server Data Warehouse and Microsoft Power BI technology. The organisation was keen to formalise its dare warehousing strategy and, after analysing potential solutions to its business challenges, Lowe says TfGM selected a cloud-based approach that fitted with the organisation’s broader IT strategy:

We’ve got a cloud-first principle and I really wanted data warehousing as-a-service. So we went out to the marketplace and looked at a number of different vendors and considered what would fit our needs in terms of our cloud strategy and out cloud principles, such as pay-as-you-go and infinite scalability.

A data-led approach

TfGM selected Snowflake and is using the software specialist’s cloud-built data platform. The organisation worked with data-warehouse specialist Crimson Macaw to improve its wider technology stack. As well as Snowflake, TfGM has adopted AWS using AWS Lambda functions, Matillion ETL and Tableau online. Lowe says this combination of cloud-based technologies gives his team insight into issues across both the Metrolink and the wider transport network in Greater Manchester:

We can understand where congestion is on the network and what the capacity is. We can also understand the wait times of trams and punctuality, which helps us improve efficiency, so we can see where the wait times are too long, and then we can try and do something about it. It really helps to move the tram network a lot quicker. We do modelling on the system as well for our transport strategy, so we can do a lot of live models and we can analyse the impact of any developments on the transport network and journey times.

Lowe says his team combines its own transport information with other external sources. The result is a data-led approach to transport modelling that allows the business to identify how modifications to certain elements of the network could have an impact on congestion:

We’ve got a lot of social and demographic data in there, such as census data and information on how people travel. We do a lot of ‘what if’ scenario planning for what we call transport modelling. And we're starting to look at bringing in real-time data from the highways network, and also from the bus network as well. That then allows us to understand where bottlenecks are, journey times, wait times, the impact of road works and the impact of changing the timings of traffic signals.

Lowe says big data allows his team to not only see the status of the network but to also understand what happens on the network when there's a problem and to see the impact of interventions. He gives the example of a recent early-bird promotion for contactless tickets – by encouraging people to travel off-peak and to save money, his team could use the platform to investigate the change in passenger behaviour:

Are more people travelling just before the peak to take advantage of those cheaper tickets? If so, great – it's worked. If not, we can we can tweak the model accordingly. Everything we do is data-driven. And the cloud-based platform allows us to provide mobile-enabled insight to the business – our chief operating officer can use it when he’s having discussions with the mayor; he can show him what's happening on the transport network.

Useful AI

Further developments are planned. TfGM is currently exploring how it might use machine learning and artificial intelligence to control traffic signals and traffic junctions. That programme of work – which is at a pilot stage right now – would allow the organisation to assess, for example, how re-routing traffic after an accident might help to reduce road congestion. Lowe says this pilot is all about finding useful ways to exploit TfGM data:

It’s about using digital principles: start small, find the use case, find a need and then build it up as you go along. We've got the foundations and a platform in place and now we can build on top of it. Snowflake just gives us the flexibility to do that.

When it comes to best-practice lessons for other IT leaders who are thinking of implementing a cloud-based data platform, Lowe says early proof-of-concepts – both in terms of the initial project and further roll-outs – are likely to pay dividends. He says the cloud provides a great platform for CIOs who are looking to run data-led modelling:

You can focus on delivering the business value, and not focus on the tins and the wires – and that, for me, is a great thing. Other IT leaders need to show the art of the possible and get people really excited by it – and then build on that platform for the rest of the business

A grey colored placeholder image