Internet technologies promise us a great deal: the possibility of frictionless enterprise, improved customer experiences and greater transparency are the oft discussed topics. The current fracas touches upon a topic I have often seen as one of the greatest prizes: the ability of service providers to aggregate data to provide greater insights across multiple dimensions. Here is what I mean:
Let's assume you are a service provider collecting accounting data across a couple of industries from thousands of businesses. Now imagine that you could analyze the content in the general ledger codes. This is the sort of thing that a forensic accountant would do when answering questions raised in a tax investigation. It is the sort of thing I used to do many years ago using clunky spreadsheets.
The problem was that my analyses were always flawed because I only had a limited data set with which to work. I was never sure whether the patterns I saw were truly representative of what I needed to see. Even so, they were better than nothing and valuable in supporting client needs.
The new service providers are in a much better position because they have a much broader reach and I can envisage many good uses for that information. For example, what might we learn about insurance premium patterns? What about understanding patterns of gross profit across different business SIC codes? What about testing DSO efficiencies?
I know of at least one provider who was able to understand the unfolding 2009 recession as a result of general trading activity levels they saw in their servers.
Now think about how these relatively simple data could be augmented with non structured data - the so-called 'big data' topic. At this level, all sounds good with endless possibilities in sight. So how does the current fracas impact this ideal?
The privacy problem
The NSA/PRISM case is raising many concerns about privacy. This has always been a thorny problem because we (as in the general public) can never be 100 percent certain how companies use the data they collect.
One school of thought believes that we have implicitly lost our right to privacy. Robert Scoble goes much further. He argues that 'we' don't care about privacy. From Facebook:
Why is the PRISM story going to disappear within two weeks?
30,000+ people die in cars every year in the US. We don't care. We get into cars every day. We really don't care about this threat. Even though the consequences are MUCH worse than anything PRISM does. The good of driving outweighs the bad of death, etc.
PRISM hasn't killed anyone (that I know of) and possibly has saved us from harm. We really don't care about our loss of privacy (I see it every day as I look around at everyone using grocery store loyalty cards, credit cards, and hand over a ton of data to big companies from Casinos to Hotels to social networks).
We are still headed into an Age of Context where systems are going to use our private data to see new patterns and help us live our lives and companies will use that contextual data to serve us better. And governments will continue to use them to spy on us (aka try to catch terrorists). Either way, the good far outweighs the bad. So we will continue to push into the future. This week will just be a slight speedbump.
Anyway, where were you all when cameras started appearing everywhere taking photos of people running red lights in San Francisco?
Truth is, we give away our liberties and privacy all the time. The Golden Gate Bridge just turned on cameras that capture everyone's license plate. Do we care? No, because now getting across the bridge is one to 15 minutes faster.
He has a point. The question is not therefore whether we are giving away our rights but the consequences. Euan Semple talks about the perils of assymetric openness:
...being more open about as much as we can as individuals and organisations is a good thing for all concerned. Whether it is the benefits of “writing ourselves into existence” or making ourselves and our organisations more accountable it is attractive as an ideal.
All of this is only OK if everyone is as open as everyone else! Yesterday’s fuss about the NSA collecting data opened many people’s eyes to the darker aspects of our new found technological ability to share. (Worth reading Mike Arrington’s take on this). The fact that they can be collecting data about me and extrapolating meaning from it without my awareness or ability to react is the problem. Asymmetric openness just doesn’t work.
Business already understands this. It is one of the reasons why many CFOs are circumspect about adopting cloud technologies. I believe those concerns are unfounded - or at least I did. I genuinely believe that vendors will act in our collective best interests. It doesn't make logical sense to behave in any other way because there is economic value to be extracted for everyone.
Ray Wang argues for better approaches to privacy:
Many would like us to believe that privacy is dead. Yet, privacy is a societal choice — it is only dead if we allow it to be. We should insist that businesses and government agencies offer choices to engage in both offline and online models. This may result in a rebalance of how much privacy we are willing to trade for convenience and lower cost.
The dark side
Many in the European Union have expressed concern about the ability of the US Government to wield the Patriot Act outside their shores. To date, I have tended to view those concerns as overstated both in principle and practice. I cannot for instance envisage how foreign agencies could realistically dig into masses of business data across numerous systems with any prospect of successful outcomes. I'm not sure the technology even exists although I am happy to be corrected. But then as the NSA/PRISM story unfolds, it creates doubts and uncertainties that cannot be ignored.
If an over-reaching government really can peer into our business data in the name of national security then I have to wonder whether we have just taken one giant step backwards on the road to business progress. I sincerely hope that is not the case although some already agree.
UPDATE - here is a very good and easily digested story about how metadata can be used...and abused.
Featured image credit: © maxkabakov - Fotolia.com
Image credit: Maurice Hermans