Dynatrace fills out Grail to make better sense of data

It was only at the backend of last year that Dynatrace launched Grail, an observability tool that aims to overcome cloud complexity through the the use of AI and analytics, and already the company’s Bernd Greifeneder, SVP, Chief Technology Officer and Founder, is extending its capabilities, particularly in areas that increase the granularity of the context in which data exists, as hinted when Grail was first announced. 

The extensions come at a time when the applications and services surrounding the data that carries any direct business value continues to grow. It is now said that these supporting services result in data that is some five times the size of the valuable data. This is now approaching the point where it can become counter-productive, in as much as there is every chance that management complexity becomes the cause of operational problems rather than a solution to such problems.

Already individual applications and services require management services to ensure their operation. Now there are management services to ensure the smooth operation of the existing management services. This can end up obscuring the context of the data.

Dynatrace is now taking real strides to pull together its tools for the management of real data – so that the underlying context can be enriched and made more explicit, allowing real business decisions to be made with some confidence. But it has to be said that this will only benefit users that are already users of Dynatrace in managing their business processes. There still lies the next big hurdle, managing across proprietary boundaries and divides.

Getting democratic

At a recent conference in New York, Greifeneder formally announced the first batch of extensions and additions to Grail. Though, as usual, he was coy about looking too far into the future, it is clear that he expects there to be more to come in the quest to increase the clarity available to business decision making, while avoiding the likelihood of the inevitable complexity actually getting in the way. He said: 

We want to ensure that you can analyze your observability and security data together with business data, obviously all in context, but also tangible to your domain, in your language, how you speak it and access it. We have completely rethought how to work with data and have come up with the new Dynatrace experience.

First of the new components is a new design system aimed at resolving an old problem, providing a way to coordinate different teams across departments so that deeper business insights can be derived through bringing together the different perspectives of those teams. 

The target, therefore, has been to develop new ways in which to democratize the existing Dynatrace user interface, making it less trivial, more accessible, and ultimately open the door to better collaboration. The new design system aims to enable the creation of a UI that accommodates both a cohesive, consistent experience on one hand, and allows for customizations and personal preferences on the other. 

This has led the company to confront and overcome three common pain points where reality trips up design objectives, with often trivial little issues. A classic example is working together with data rich interfaces and standard typefaces. Here it is often difficult to decipher small but important differences between, say, a zero and an O, or an l and an i. These are often the seeds of major mistakes and distractions. The Dynatrace solution has been to develop its own custom-engineered typeface where digital clarity is more important than human appreciation. 

The second issue it addresses is providing the ability to better focus on the subject at hand when navigating around many different data sources, when business insights are being created. The solution has been to build new flexible interface components with embedded actions, allowing users to get to the deep insights, whether it's a data point in a table cell or maybe a marker on it, faster.

The third issue is that teams still work in silos, and when any problem arises they immediately start to divide, with each looking to conquer the problem their way. So they start to create their own charts, their own reports, in their own spaces. Not only do they not communicate, but they duplicate work. The goal of eliminating these silos has been achieved by re-engineering its chart technologies so that now users can overlay multiple chart types. This can bring teams closer together when working on one issue.

Automating queries 

Creating a new dashboard can obviously offer a great deal of new potential in terms of data display but, as Greifenender observed, it also creates new problems. He said: 

Creating a dashboard is easy, but only if you have all the data that you need to put behind the charts. And it turns out in reality that this is a complex task, because you need to find data, you need to clean it from errors in the data, you need to filter it, you need to transform data types into a metric that you can use in a job. And of course, you want to present data coming from different sources in a way that they are connected, so that people can understand dependencies.

Till now those tasks have been a job for data scientists, and he suggested they can spend as much as 80 to 90% of their day on these easy tasks, rather than building advanced models that help predict the future for a specific business. Cutting away this roadblock is where the next addition to Grail comes in. Known as Notebooks, the goal is to further democratize the ability to work with data.

Notebooks is an application design to create data-driven documents for customer analytics. Coupled with Grail, it provides users with a tool to query all their data, execute custom code and add annotations or context all the time, using the new query language introduced last year. Querying has been automated to the point where queries are created automatically once a user clicks on a data item, meaning that many will never even need to learn the query language itself.

Notebooks are really a new way of working with observability and security data. And what excites me most when working with notebooks is really how interactive you can be working with data in charts. You click on something and the query which helps you to get started, you can get to a lot of success without even learning the query language. This is expected to make it much easier and faster for users to perform tasks, such as documenting and analyzing a production incident, forecasting future infrastructure resource requirements or evaluating the impact of a marketing campaign on revenues. Greifenender said: 

At the end of the day everything thinkable, on top of data that is in Dynatrace, becomes doable.