The Friday Roast

Over the holiday period - which for us is still kinda running - we spend a good amount of time doing a pile of maintenance jobs. diginomica is a hefty site and there's a lot going on so we like to take this time as one where we clean up as many of the messes we've created during the year, get things running just that little bit better, evaluate some of the functionality we've built in, address user niggles and so on. It's a long list.

One of the things we did mid-term last year was to have an all out purge of broken links. We had a LOT to clean up, around 500 and most of those were image related. Imagine our surprise and shock when our engineer told us that five months on from the last clean up we had accumulated 1,600 broken links. Well over 1,200 of these related to image issues, a constant source of problem and, in truth, mostly our own fault. Fortunately, we were able to clean those up relatively quickly with some magic dust our engineer sprayed onto the site. That left us with several hundred broken URL links.

N0w - anyone who know anything about Search Engine Optimization will tell you that Google's algorithms don't take kindly to broken links when assessing your Google rank and so keeping it clean is something any site owner should treat as a priority.

Apart from our own sloppy internal linking, which accounted for a modest number and for which people have been taken out the back and given a good thwack with a cluestick, there were an alarming number of broken external links. These are links we have added based upon the URL information contained at other sites. And we have a LOT of links since we are firm believers in the concept of 'sending them away to bring them back.' The idea is simple: giving a link to something that informs our thinking is a way of demonstrating that:

We don't know it all
There are super smart people who help us get smarter
It's good manners to acknowledge the work of others rather than claiming it as your own.

Much of the initial impetus behind the blog movement of the mid-2000's was predicated on copious linking. It made a huge amount of sense and helped proliferate interesting ideas and stories.

Sadly much of that practice has disappeared and especially among the large scale media. Nowadays, some media doesn't bother to link at all even when it is obvious they are creating derivative works. In one sense I can understand those practices. Links break and cleaning them up is a soul destroying task. My view was that since the links are external then we should not waste much effort on them. Our engineer was a bit insistent on that point so we cleaned them all up. Hundreds of them.

Along the way, I was appalled to discover just how many organizations are incredibly sloppy at cleaning up their own mess. Several UK government sites just don't seem to bother. I guess Google rank doesn't matter to them so much but simply hauling down pages that have links to important research documents is no joke. similarly, I discovered that some commercial sites have wholesale rewritten their web presence without thinking about links. Some fairly high profile media had done similar things, yanking pages without any attempt to redirect the reader to a new location. Taken as a whole, I was bog smacked to find just how sloppy and careless a wide variety of operations have become.

What these same operations don't seem to grasp is that by being sloppy on this topic, they're eroding their own value. They may not have noticed. Some may not be watching for this issue. Still others may not understand the importance of clean links. There is plenty of guidance and while we have taken a stringent approach to this topic, some advisors claim that it is not necessary to be as rigid as we are.

Our experience has provided us with a very valuable lesson. From here on in, we are treating link maintenance as something that has to be reviewed regularly, not just when there are other maintenance tasks to hand or when we remember it. We believe the investment in doing that work is worthwhile because while we can't do anything about what other sites do, we can do something about what appears on our site and if that ensures the user experience is as good as it can be then that's a no brainer.

404 spreaders? You've been roasted!!