Last night's Australian Census was nothing short of a debacle and IBM is firmly in the firing line.
A sure fire way of guaranteeing an IT project will fail is to assure the public there will be no problems. So it is this morning for the Australian Bureau of Statistics whose management, and the country's government, are surveying the smoking wreck of the country's 2016 Census that was supposed to have been run on Tuesday night.
The public servants' aims were bold, getting two thirds of Australians – about 16 million people – to submit their forms online would save the government hiring over fifty thousand census collectors and deliver the 'efficiency dividends' that have seen the agency's budget whittled away over the last twenty years.
Originally the ABS had intended to carry out the 2016 using its own virtualised platform but late last year re-established their long standing relationship with IBM in what the agency called a 'limited tender' for 9.6 million Australian dollars to provide an online platform running on SoftLayer. The rationale for outsourcing was that a third party provider would be better placed to deal with expected demand spikes on the night.
The ABS were confident about IBM's SoftLayer capabilities, an ABS spokesman told the Melbourne Age two weeks ago that the site could handle:
1,000,000 form submissions every hour. That's twice the capacity we expect to need...there is plenty of reserve capacity to cope if more than 80 per cent of Australians choose to complete the census online.
By 7.30 last night it was clear that confidence was misplaced as the website crashed and social media users were entertained by the ABS's Twitter account gleefully autotweeting messages encouraging people to complete their census online in response to the hundreds pointing out the service was down.
But check this out from LinusTech Tips in a nattily titled: The Australian Census website was DDoS'd 4 times last night and Australian media is full of shit
The ABS (Australian Bureau of Statistics) spent hundreds of thousands of dollars stressing the systems for 1 million forms an hours. The thing is there is 23 million people in Australia, and there is an average of 2.6 people per household, so there would be 8.8 million households, and what time would you complete the census? I know I completed it after tea at about 8PM, I am speculating that most people completed it sometime after work and before they go to bed or start binge watching TV. Assuming 90% of 8.8 million decided to enter it between 5PM and 9PM, 7.92 million households tried to enter the form and that is close to 2 million per hour, just under double what they tested it for, which was showing when I completed my form. When I was completing my small part of the form, I got 3 errors saying my information was not uploaded, ultimately it didn't effect me since my data did get submitted, but the cracks were showing.
At 10.30pm the adults had wrested back control of the agency's Twitter account, stopped the autoposts and posted an apologetic message that the site had, indeed, crashed. Along with the agency's credibility.
In the cold grey light of the following morning, a still feisty Head Statistician went on a media offensive to blame denial service attacks – the cyber equivalent of saying 'the dog ate my homework.' To make matters worse, at a lunchtime press conference the minister and his senior bureaucrat contradicted each other over the cause of the debacle.
Further contradicting the denial of service narrative is the lack of any evidence of unusual traffic across Australian networks at the time, although even if there had been a DDoS launched against the Census website one would expect IBM has the tools and skills to mitigate it.
Now the Australian Bureau of Statistics has a major credibility problem as it spent the last three months arrogantly dismissing concerns about security and reliability of the online Census. One of the lessons for any executive in this debacle is not to be so publicly contemptuous of informed critics in the way both the Australian government and bureau of statistics spokespeople have been over recent weeks.
The bigger problem though is for IBM, the company's Australian reputation is already tarnished by the specular collapse of Queensland Health's payroll system in 2010 that cost taxpayers 1.2 billion Australian dollars and has seen the company banned from any state contracts. This debacle won't help their sales teams.
For both IBM and the Australian Bureau of Statistics, the biggest task out of last night's Census failure is going to be in rebuilding their shredded reputations. One thing for sure is arrogantly asserting there are no problems will no longer cut it for either reputations.
The humiliation of the Australian Bureau of Statistics is well deserved. The agency's hubris in the run up to last night's Census was nothing short of remarkable. At the DefCon conference in Las Vegas last week I was asked by a number of security professionals if I shared the ABS's confidence about their online defences. I could only shake my head and politely order another round of drinks.
Once again we see a government agency outsourcing a core task without having the internal contractual and project management skills to deliver the project. As consequence they fail to achieve the savings or efficiencies touted by the ministers and departmental heads who should know better than to gormlessly accept the assurances of sales people looking for fat commissions.
The frustrating thing with the Australian experience is once again few of the managers responsible for these contractual debacles will be held accountable for their incompetence and one suspects that nobody at the Australian Bureau of Statistics will be politely shown the door as a consequence of last night's debacle.
For these larger companies like IBM, which have at best a spotty track record of delivering these projects, it's hard to see how their business model can survive in an age of keen young startups offering far better solutions without the massive implementation costs and risks.
Overall, the 2016 Australian Census debacle is another lesson to managers about blithely expecting large IT implementations to go as seamlessly as the sales folk promise. Taxpayers also should be demanding better from their public servants and elected representatives. Technology can never solve hubris or incompetence.