‘Streaming data, but screaming staff!’ - Ciena turns to Graph to solve customer issues

What is it consultants do for their clients? Let’s resist the cheap gags about taking their wallets then selling them back, thank you very much. Consulting services are a huge part of the IT ecosystem, after all, and customers plainly value haven them come in and try and model their problems and offer ways forward.

That’s not a definitive definition, so you might prefer this, from Simon Brueckheimer:

What consultants really do is try and understand the customer’s challenge and then try and offer a solution to those challenges that fit as snugly into their environment as possible.

If you like that encapsulation, then that’s good news for Brueckheimer in his role as Leader of  the Adaptive Solutions Consulting team at his company, networking systems, software and consulting firm Ciena.

Ciena has over 1500 customers worldwide, including BT, TelemaxX and BSNL. A core claim of the company is that “practically” all we consume from the Internet, Web or cloud spent some time on a Ciena platform on its way to us as users.

Unsurprisingly, Brueckheimer states that data, analysis, automation and “workflow autonomy” all inform what he and his team do. But much more interestingly, he now says he’s found a very useful new way of modelling the complex situations he finds his customers dealing with - the Graph view of data, as in that way of representing complex networks.

That’s only after those problems go through an intermediate level of formalisation, though - what Brueckheimer calls ‘signatures’:

A signature is a common word in the problem solving world, and also in the dictionary, where we read it’s any distinctive pattern or characteristic by which something can be identified.

I use it to allude to the pattern and characteristics of real world client challenges - whether business, operational or technological - that, with our consultancy expertise and skill can be naturally and ideally captured by Graphs, and that thereby benefit from all the advantages modelling problems this way have over the relational one.

Why Graph?

As to why Ciena and the Adaptive Solutions Consulting part of it got to working with this idea of graph/signature,  Brueckheimer explains:

Our business problem was that we wanted to move into a different realm from the technological capabilities of the technologies we had been using up to that point.

There are many different problems and challenges that are data-based; understanding that information, and seeing if that information was adequate for solving a business problem, is one challenge - and if it isn’t, determining if there is a method for one can analyse that information further to get the answers is another.

Brueckheimer posits that the information needed to perform this analysis can often be historical, and doing so doesn’t have to be done urgently. This, after all, is how businesses do monthly budgets and financial forecasts. But there is another set of problems in business that are much more time-dependent, such as when a user is interacting with data and wants to do a what-if analysis, or where we’ve automated some process and require to do some calculations on the data.

This latter class of time-sensitive problems he defines as ‘transactional’, and is the problem set his team have found graph technology useful (among others). He offers an example: a (physical) network problem:

Network elements ‘know’ each other, but most network management systems just stitch all the pieces together and can only be interrogated ‘piecemeal’ - plus, there are thousands of ways of doing that interrogation, with variable results.  But a Graph representation ‘stitches’ everything together for ‘free’, and gives you more information about those relationships.

Another example Ciena has found useful is in tracking automatic circuit switching problems, where there is no longer an issue of finding the ‘best’ data relationships, with Brueckheimer also sees applications “naturally’ talk Graph, too, with what he styles as ‘real-world abstraction’ being another useful bonus of this way of working. Hence why he uses his metaphor of capturing ‘signatures’ is how he sees graph problem solving being a practical tool:

The art of applying Graph technology is to look for natural ‘signatures.

Admittedly, all this sounds pretty abstract so far. But Ciena says this genuinely applies to urgent, practical issues in what he dubs the nightmare of dealing with “streaming data and screaming staff”. He explains:

Network elements are noisy, they fail and give out alarms, while monitoring and forensic tasks require both synchronous and asynchronous event recording and root cause analysts. But fixing these issues is often manual, there can be a lot of both physical and abstract structures and it can be hard to visualise, let alone trace, what’s going wrong. That’s why network engineers often find themselves wading through history to solve the problem, or fighting a kind of ‘unfolding present’.

But using Graph - in this case, Neo4J , he says, automated root cause analysis tools can start to become possible, as one of its properties is to compute the number of things connected to the thing we’re interested in - so giving the observer the right number of distinct elements, with that work getting done in linear time, too.

Adaptive Solutions Consulting staff can then use Brueckheimer’s ‘signature analysis’ methodology to quickly see just what type of event the network is having - with this automated analysis being just one emerging use case of Graph-style modelling that are being investigated by the team, he adds.

Summing up the impact of Graph on his company, for this consultant:

Natural world problems, particularly in the telecoms space, have ‘signatures’ that mean they are naturally a good fit for graph technology. And with Graph, you reflect real-world properties as relations and doing so you are coding as scheme the minimum of what the application might need - and can do a lot of calculations itself, giving us a lot of flexibility.