Beyond project failure - into Q/A independence

Profile picture for user jreed By Jon Reed April 7, 2014
Most of the easy-bake recipes for avoiding project failures are stale. One overlooked factor? The impact of independent advisors on enterprise projects. Here's how one of them does it.

Holding tight rope
Enterprises evolve, but project failures remain a disconcerting constant. Tagged 'failure' articles from my newsfeed shows the ERP heavyweights are joined by other flavors of failure - from retail to startups, from security meltdowns to digital transformation chokejobs.

It's naive to think that as companies move away from a multi-year 'waterfall' mentality, that project failures are behind us. It's equally absurd to propose a magical remedy.

But most of the easy-bake recipes for avoiding project failures are stale. One overlooked factor? The beneficial impact of independent advisors on enterprise projects, whether on-premise or cloud. I've covered this topic before from the vantage point of vendor selection advisory and hands-on experts, but not from the angle of a higher level independent who provides ongoing Project Quality Assurance (I'll call this Q/A, but it's not the same type of Q/A as hands-on patch testing).

Over the course of his extensive IT career, Michael Doane has seen the advantages and pitfalls of that higher-level Q/A role firsthand. He's done the expert witness thing on project failure lawsuits, and along the way, he's developed maturity assessments that help customer to evaluate ERP projects, with an eye towards significantly greater business user buy-in, via the involvement of  'process owners' in super-user networks.

On a recent video hangout, Doane and I dug into the meat of this topic. We charted out the independent Q/A role, and also laid out other keys to project success that are frequently overlooked - at the customer's peril.

Why does the independent Q/A specialist matter?

As Doane sees it, a key role of the independent Q/A advisor is to provide periodic gut checks from a different perspective than the main systems integrator (often called the 'Prime'). A classic example? The SI proposes an expansion in project scope. But is this expansion necessary? Doane:

The main thing is for the systems integrator to know that someone’s watching. Not like an auditor, but in quality assurance. The client may not really know, 'Is this a good idea? Is this a bad idea? Are they pulling the wool over my eyes?' Your quality assurance person can say, 'Actually, that would be quite valuable, and you should give it strong consideration,' or they might say, “I don’t really see the value. Make them demonstrate it to you first'.

The obvious objection is: wouldn't an opinionated advisor make a project more difficult to manage? And what if the prime vendor perceives the independent as an adversary? Doane doesn't accept those excuses. It's not the advisors' job to take sides or demonize the SI. It's the advisors' job to take positions informed by the interest of a successful outcome:

Whenever I’ve gone on a site to do quality assurance, the service provider in question is hostile to me early on, again, on the presumption that I’m going to be looking over their shoulders and doing 'no, no, no' to the client. In point of fact, and they’re almost always pleasantly relieved, when in a given meeting we’re going through things, and I turn to the client and say, 'Your systems integrator is right. You shouldn’t be cutting those corners. It’s going to hurt you.' [My job is to] demonstrate that my role is to help them have the most successful outcome as possible, and that cuts in both ways.

Doane does caution against putting too much faith in the consulting firm as a so-called 'trusted advisor':

Why independent advisors matter to enterprise projects - Brian SommerBrian Sommer Here's my deal …Jan 14

When I was covering services, I would laugh purposefully every a consulting partner would say to me that old song, 'Michael, we are our clients 'trusted advisors.' I would laugh, and say, 'No, no, no. You always have more to sell, and the client knows it. They’re not going to have the kind of trust in you that they have in me, because I have nothing more to sell. They’ve already bought me. That’s it. That’s all I’ve got in my kit bag. I’m not a systems integrator. I don’t have a raft of consultants on the bench, none of that.'

Finishing on-time is not the same as delivering value

Doane is infamous for rejecting the notion that a project delivered on-time constitutes a success:

One of the more amusing moments was when I was being deposed once by an attorney. The attorney made the mistake of reading my books and thinking he knew something about SAP. He said to me, 'Mr. Doane, wouldn’t you agree that a project that’s completed on time and on budget is, by definition, a success?' I said, 'Absolutely not. I’ve seen a lot of projects on time and on budget to no appreciable benefit to the client.'

To get a better handle on the actual risks and benefits, Doane developed a maturity assessment methodology (here's an overview of the SAP version). As part of the self-assessment, customers grade their project maturity based on a number of factors on a 1-10 scale, with 10 being the highest score. Scores under 7 mean there is reason for concern/need for improvement.

Doane shares that the state of end user competency is typically graded at 5.3, which is, in his words, 'meh'. The ability to measure results is 5.3. Business and IT dynamic is 5.8, which he characterizes as  'Well, we’re not killing each other, but we don’t have lunch together.' The state of the applications is 7.3, which can be misleading. Doane explains:  'The clients say, 'Hey, the applications are working. Maybe in theory, but with those other elements being so weak, no, the applications aren’t working.'

The downside of these stark assessments? Paralysis can set in, as in: 'Do we have to retrain everybody?' The answer is: no. Doane recommends creating a super-user network, which in turn has a documented positive impact on end user productivity and morale. Doane's goal? To rectify an imbalance he found gathering data at a firm called Performance Monitor. Those surveys found that project sponsors gave projects the highest satisfaction scores. Business users gave them the lowest. And that's a recipe for disappointing outcomes, if not outright failures.

Final thoughts

Failure rarely has one culprit. The same is probably true of project success. But there is an underlying factor that shows up repeatedly: successful projects are run by customers that have claimed autonomy over their projects, rather than handing over the keys over to an outside partner. Doane calls this 'client ownership'.

In this diginomica series, I've emphasized the value of independents, but the savvy use of independents is just one of the client ownership elements customers should pursue. A working list includes:

  • Smart use of independents, not only during vendor evaluation but throughout the project lifecyle.
  • Active involvement in users groups for peer-to-peer education and advisory.
  • Use of maturity assessments to identify gaps in skills, method and IT/business alignment.
  • Increased use of business process owners and inclusion of business leaders in constructing meaningful KPIs.
  • Aggressive investment in training and skills development, using the latest approaches for collaborative learning - perhaps supported by the creation of centers of excellence around vital internal skills such as business intelligence.

(All but the last aspect, on centers of excellence, are covered in our recorded discussion - Doane's views on centers of excellence can be found here, and are also detailed in his SAP Green Book).

We can have a worthy debate about just how 'independent' anyone is in this marketplace. But the high stakes of enterprise projects clearly call for multiple viewpoints, and not just from one service provider. Failure is no more avoidable than success is guaranteed, but we do know this much: course corrections are best made when the iceberg is not yet visible.

I left out a lot of interesting content - here's the full audio from my 50 minute talk with Michael Doane (video version here, downloadable version here):

Image credit: Holding tight rope © igor -

Disclosure: Michael Doane and I co-authored and published a book together in 1998 (The SAP Consultant Handbook). We do not have a current financial relationship. Although Doane's views apply to a range of enterprise projects (particularly ERP), the video discussion is mostly drawn from his work with SAP customers. SAP is a diginomica partner as of this writing.

End note: This piece is part of an occasional series I am writing on the impact of independent advisors on enterprise projects.