Friday Rant - why Agile development is a fad, not a proper engineering approach

Profile picture for user cmiddleton By Chris Middleton September 11, 2020
Summary:
A UK standard bearer and assurance expert takes a sceptical stance on Agile development…a bridge too far?

ranting
(Pixabay )

Agile development is a fashion, but that is not necessarily the same as designing and building systems that work and are secure by design. That’s the view of Nick Tudor, CEO of Malvern, UK-based D-RisQ, a specialist company in the critical discipline of proving that software-based systems actually do what they are supposed to do. He says:

Agile is a fashion. It was a great piece of work done by some academics, mostly in the US. They wrote a document called the Agile Manifesto [2001], but not many people actually read it. And among those that did, quite a few of them didn’t understand what it's all about.

The idea is essentially that there are some attributes on the ‘left’ [a notional column] and some on the ‘right’ and we prefer the ones on the left to the ones on right, but that's not saying that we're going to throw away the stuff on the right! We just value one set more than we do another at certain phases of the programme. Unfortunately, this gets rapidly interpreted as, ‘We just need to get some code out’. [...] That’s not the way to do it.

In a world of intense commercial pressures to move swiftly and capitalise on new business opportunities, faddy mantras such as ‘fail fast and move on’ and ‘move fast and break things’ abound. But in my own view, there is surely a risk that developers simply fill the world with broken things. 

Look at the lamentable track record of software development in the public sector, for example. There, poorly specified projects are often forced through quickly and rewritten on the fly, with mission-creep dragging them far beyond what they were conceived to do. The end result is systems that are over-budget, absurdly complex, late, and often built for a world that no longer exists – if it ever did.

Other projects have abandoned proper development principles for a quick fix – in vain. No one looking at the Kool-Aid guzzling, mates-rates, mould-breaking public sector in recent years would see anything different.

That’s my view, not Tudor’s.

But in his business, which embraces such areas as robotics, aerospace, and defence, filling the world with broken things simply isn’t an option, given that some of them may be airliners full of people, or autonomous drones that humans need to be certain won’t collide with each other in the crowded skies above cities. And he certainly knows what he’s talking about here as, among other things, Tudor helped write DO-178C, the software standard for the entire aviation industry. He continues:

What we really need to be focusing on is, by all means get to the code, but you must have gone through the first step of the process that says, ‘What is this code going to do for me in the first place?’ If you don't do that, how have you got any basis from which to say, ‘I can verify this code satisfies that particular requirement’? How do you know, if you haven't written that requirement down and agreed it with the customer, that the customer is even going to accept it?

There is no shortcut to this. You can you can either do the process that gets you to the code really quickly, then go back and check what the customer actually wants and spend a huge amount of time arguing about it – having spent an awful lot of your budget to get to that point. Or you can spend the time and effort up front to get those things sorted. It's understanding what these processes are and implementing them properly.

I will add that the Agile approach, if you go back and look at the manifesto, was originally designed to be used by highly coupled teams that sit around one desk, so there are roles for each individual and they can talk to each other – ‘I own the requirements’, ‘I own the design’, ‘I own the code’. But I've never seen it work very well in large teams.

A bridge to somewhere

For Tudor, coding, software design, and development should all be considered as a branch of engineering – as a practical implementation of mathematics in order to build safe, reliable structures that do exactly what they are supposed to do. And that demands the application of Formal Methods, he says:

In my view, this is engineering. There's no excuse for not doing the job properly – and you should be held accountable.  [Formal Methods] is a very difficult discipline. However, if you can use it, and that really means from an automated perspective, then you can get rid of a lot of subjectivity in reviews, and it forces you into a discipline of understanding exactly what it is you want from the system. It helps you to meet standards.

Time for a useful word picture to make the point: 

The analogy of a bridge is something I use a lot with people who don't understand software engineering. If you came to me and said, ‘I want a bridge’ and I went ‘Great, that’s £250 million pounds, here are two planks’, are you going to be happy with that? The answer, hopefully, is no, but as far as some developers are concerned, that's a bridge.

If you're building a bridge you have operational requirements. What you'll then do as a contractor is your specification, and you will agree that with your customer. Once that's done you can move onto the next step, which is doing the actual design. At which point you'll do a whole series of models, analysis, materials research... and assure yourself that the design is going to stand up. Otherwise it's going to be embarrassing when you start pouring concrete, shipping all that steel, and welding it together. 

That's an engineering process. Well, replace operational requirements with system requirements for software, and replace bridge specification with software requirements, bridge design with software design, and concrete and steel with source code. I can do exactly the same verification process.

Tudor shares insights from his long association with aviation, among other sectors – in this case an unnamed national air traffic system some years ago:

This guy decided when he put out for tender that he was going to do all the safety-critical stuff that's needed for an air traffic control system – you'll be relieved to hear. But he also decided he couldn't afford what people were saying actually needed to be done. So he opted to integrate a lot of off-the-shelf software that he just bought in, and then retrofit all the safety and security onto that. And after two years of his one-year plan he had spent all his money twice over.

He does acknowledge that keeping pace with change and business demands for new toys to play with is difficult, but that doesn’t mean developers can’t take care in their work.

There's still some basic stuff that can be done. There's no doubt that if you understand the tools you're using, and the limitations of those tools, you will have a better way of developing software. And if you invoke proper processes and good standards behind them, including the way in which you write requirements and do your design – as well as the coding itself – then you will save yourself an awful lot of heartache in the future.

My take

Wise words in a world of broken things, in which more and more people cut corners, fail to understand what they are really doing, and end up with projects that are the opposite of the nimble, edgy, ‘world-leading’ innovations they thought they would get. 

Sound familiar?