PagerDuty CTO - ‘Generative AI focus should be about minimizing collateral damage’

Vendors across the B2B software market are making their generative AI pitches, following the release of ChatGPT late last year, which showcased the potential of Large Language Models and how neural networks could be utilized to create content. The past few months in particular have seen software vendors describe how the world of work is going to be transformed by generative AI models, which will miraculously remove much of the menial tasks and grudge work that knowledge workers have to do today. 

However, the risks and concerns have often been downplayed, with a lack of authenticity from some when it comes to the challenges around a lack of regulation, data protection, models hallucinating and being unable to ‘unlearn errors’, as well as a certain amount of trivializing when it comes to happen to the skills base in the market. 

As such, it was welcome to speak to PagerDuty CTO, Tim Armandpour, about how the vendor is thinking about the impact of generative AI, particularly in terms of minimizing damage over the long term. Armandpour’s thesis is that, at present, the generative use cases companies will be working with will likely need some course correction over the coming months and years and that the focus should be on identifying where issues arise and effective adjustments made. A lack of understanding where problems are, means that companies could end up with more serious data-related issues down the line, which in turn could negatively impact a company’s reputation and revenues. 

Armandpour says that six months ago the hype cycle was “real” and that AI was at the center of every conversation, but that there was a great deal of apprehensiveness built in because many didn’t know what it meant in real terms. Now, however, there’s a level of acceptance and the conversation amongst peers is ‘how do you protect from misuse?’. He says: 

How do you actually create an environment for your employees, your team members, but also your customer environments. where all this can work in a fantastic fashion, but in a safe and responsible manner? Now it’s focused on honing in and bolstering the guardrails. How you choose to engage with it, how you choose to adopt it - data management and data privacy comes up a lot 

If I'm an employee, I have my laptop - nothing can really stop me from using ChatGPT. I'm told, based on policy and the papers that I signed, that I will not share private information. But it might leak out. 

So if you think about ways you want to protect the IP of a company to start with, in that very basic use case of just ‘copy and paste this and put it over there’. That's extremely challenging. That's making a lot of people squirm in their seats. 

Employees and customers are getting more and more educated, and are acknowledging that handling data in large language model environments requires a thoughtful approach, he adds. Governance is becoming a real concern. Armandpour says: 

A pattern I'm seeing is people starting to ask each other questions in the CTO community. Everyone's investing in some manner in this next wave of technology.

What are you actually bolstering when the output of a thing is not exactly what you expected? How are you monitoring that? And how do you detect the thing that's drifting, when hallucinations come up? 

The challenge is, it's already out there. It’s not like code that you can just roll back. It's now baked into the model that you're depending on. How do you unwind it, rewind it, those techniques are still being developed and evolving. 

Zero control

This line of thinking is something that has been raised by other leaders in recent months. For example, BT’s Chief Digital and Innovation Officer recently highlighted that whenever she’s asked the question about how a foundational model can ‘unlearn’ something that’s wrong, no one has been able to give her an answer. 

‘Polluting’ a company’s foundational model is hard enough to manage, but it’s even more difficult if data is being placed in a model provided by another organization. Interestingly Armandpour notes that PagerDuty could actually play a role in detecting anomalies and when models behave in a way that drifts from a company’s desired outcomes. He notes: 

That's that next wave: I detected a thing, it didn't hit the expected case, which is not good. How do you, as an organization, whether it's the people or through automation, start to think about responding to such a thing? 

That's actually interesting, where some of these use cases start to come up across our customer base. They’re asking, can we use PagerDuty for that? It's pretty plug and play: you detect the thing, you're able to diagnose it, and you respond to it. It's the same playbook, it's just the context has now shifted. 

The water main is leaking. How do I stem it before it becomes a flood? That's where I think a lot of the thinking is shifting, it’s almost like containment. But to rewind and unlearn, I don't think anyone has a ready made answer for that just yet.

Minimizing damage

All of this is not to say that PagerDuty hasn’t started to develop its own generative AI use cases too. But as we noted when speaking with CEO Jenn Tejada earlier this year, the company is ‘going slow to go fast’, which is in line with Armandpour’s comments. The two use cases announced so far are: 

• AI-Generated Incident Postmortems - which aim to allow teams time and effort by removing toil and improving accuracy associated with post-incident analysis. PagerDuty will automatically create a draft postmortem using generative AI so that teams can focus on refining their learnings and action items for improving their operational processes. PagerDuty says that with just a click of a button it triangulates and collates incident data to help generate comprehensive summaries of what happened, when, how it was resolved, and key action items. 

• AI-Generated Status Updates - these aim to remove manual work associated with stakeholder management during incident response to keep teams focused on resolution. Early customer feedback of AI-Generated Status Updates, according to PagerDuty, has shown that some companies designate as many as three responders to handle stakeholder communication. With AI-Generated Status Updates, the vendor adds, drafting the message only takes one click, making it easier for the responder to review, edit and send updates to keep internal stakeholders and executives in the loop. 

Commenting on why PagerDuty opted for these features, Armandpour says: 

We chose use cases because we felt at the time it was right in the wheelhouse of what generative AI is known to cover pretty well. When it comes to text summarization and having the human still involved to verify. 

So we felt really good about that. I think that verification piece, is going to be pretty pervasive across a lot of scenarios, until you cross that next chasm of acceptance and capabilities 

Armandpour says the next phase for the company will see more holistic adoption of generative AI, focused on end-to-end journeys. For example, if an incident occurs, using the models to help pinpoint a probable surrounding radius of where the incident is happening, drill down, highlight the probable cause, and showcase the best next actions to take in order to remediate it quickly. The risk of getting things wrong in operations management is high, in terms of cost to the business, so PagerDuty is treading cautiously. 

The vendor is also focused on tooling to minimize damage. Armandpour adds: 

There’s going to be some collateral damage, you have to accept that there's going to be some. It's not that different from the world of cybersecurity and information security monitoring. It's not an if, it’s a when. It comes down to how well prepared you are and how much are you willing to invest to be well prepared you are to understand, to react to it and then ideally course correct it. 

At the end of the day, these models. in the grand scheme of things. are actually quite dumb. They're really powerful and exciting, but they’re actually quite dumb. But being able to find that balancing act between not too slow, not too fast so that you’re seen as being irresponsible, that's a weight on the shoulders of some of us. 

Especially in the interest of one protecting our company, but then also facilitating and protecting our customer base. 

My take

A balanced and grounded generative AI pitch from PagerDuty, one that I think will resonate well with buyers. Those looking for solutions in the market don’t want to be sold an AI pipe dream that they know is probably unrealistic - if it sounds too good to be true, it probably is. Going slow to go fast is a sensible tactic, when considering the risks associated with these models.