We are all creatures of habit and learning. In this guest post by Vijay Vijayasankar questions whether we really do know how to measure and interpret the right things. It’s a good question at a time when so many are relying upon data driven decision making as a mantra to slavishly follow.
In business, metrics are things you worry about all the time. Over time, most people do come to a conclusion that not all metrics are helpful. In fact, no metric is useful if you just watch it – you need to act on it, measure again and so on . But do we take the time and effort to question the so called ‘useful’ metrics ?
All through my career, I have had to deal with Billings, Bookings and Backlogs (the three B’s – or BBB as commonly known), and I am now pretty sure that these are not very useful metrics like I used to think.
In my time as an analytics consultant, I have helped design and deploy several BBB solutions for my customers – big and small, and in three continents. As a lay investor, I am always trying to find ways of comparing companies to figure out where best to put my hard earned dollars. And most recently, when I had to evaluate potential employers, I had to deal with it yet again .
At an elementary level – bookings is the net total of all orders you have, billings is money you expect from customers for net fulfilled orders, and backlog is what remains unfulfilled , but you are sure of converting to billings. Unfortunately – it is not that simple or uniformly defined across companies .
Billings – or revenue – is (mostly) reliable across public companies . They have to follow strict rules. But the non public companies don’t disclose revenue – and that made it harder for me in comparing potential employers.
Bookings are the most arbitrarily defined. The problem is that the time dimension makes it hard to define. If you sign a deal with a customer that pays you $100K every year for three years for meeting certain milestones, do you call it $300K bookings? What happens if the milestone gets missed ? Given each contract is individually negotiated – this makes it very very hard at a company level, and impossible to meaningfully compare across companies. Add to that the confusion of non public companies who use billings and bookings interchangeably .
Even within a company , it can get hard to manage business by bookings. When you have many products to sell, and you bundle stuff with complex discounts to make ‘elephant killer’ deals – there is some inevitable peanut buttering that goes with it on management reporting . Essentially you can’t make serious decisions based on bookings alone. This gets very complex on budgeting, sales compensation and many other areas. But as creatures of habit – managers still do it.
Then there is the backlog. Backlog is hard to keep on top of – there are all kinds of things like write offs , return orders etc that affect backlog . A complex portfolio and organizational structure also makes it harder to use backlog as a management reporting tool.
There are many other things that affect BBB reporting that I haven’t mentioned – it could easily fill a book. But it raises an important question – can we make good decisions based on BBB?
Some CXOs use additional metrics – like number of downloads of software, number of comments in internet, number of jobs open in the market to augment things like BBB reports. All of those approaches come with side effects too.
- Downloads – is it because software is great or because you had to issue a hundred patches ?
- Comments – quantity of comments might mean good or bad, and sentiment analysis is still fairly a new discipline
- Number of jobs in market – is it because software is great? Or is it an indication that the company is high maintenance ?
Does it mean you should not bother with measurements ? Absolutely not – you should of course measure everything that matters . But do so knowing all the limitations and nuances. And never trust one metric in isolation.
One final thought – good decisions are based on stuff that can be measured , but also on some important stuff that cannot be measured. For example – Trust. No one will argue that Trust is not a factor in any meaningful business decision. But it does not have a clean metric to measure. Just because something can be measured doesn’t increase it’s ability to make a decision better . Remember that when you run into the next person who pontificates mindlessly on some version of ‘data driven decision making!’
What do you think? Am I being too cynical from battle weary experience or touching on an ‘inconvenient truth?’
Featured image credit: © Olivier Le Moal – Fotolia.com