Years ago when Service Oriented Architectures were the next-big-thing, all the talk was about dis-intermediation. This meant that a company could sell directly to the end user, cutting out every business in between. Now the trend is swinging back and re-intermediation is in favour.
According to Shawn Rogers, Tibco’s Senior Director of Analytics Strategy, this trend is the underpinning of the burgeoning Algorithm Economy. It is a time when we are leaving the ‘T’ of IT behind and putting what he calls “the last mile of capability” in the hands of the experts. They are the ones that now take on the curation and enablement of solutions:
The people part of all of this, the domain skill – whether it’s an SOI team or channel partner – the domain skill sets that come from the people in these scenarios, these are the ecosystems that I can’t get into without the expertise of the people that are in them. I laugh when I read some of these articles that talk about how AI and machine learning are going to eliminate all the jobs and put all these people out of work. I do think some of it will evolve, I think some jobs will disappear, but new jobs will come about.
Time for re-intermediation
To Rogers, there are four disruptive drivers that are happening in the industry today: technology, community, financial or economic advantage, and total data availability, of which the last is perhaps the most important. The fact that a business has all of the data available to it puts it in a spot where it is able to really develop, argues Rogers:
If all four are happening at your company you’re going to change and you’re going to do things differently than you were ten years ago.
That change applies not only to the users but also to the vendors. They need to learn to accommodate and integrate their emerging re-intermediated partners. He acknowledges that Tibco is no stranger to this change and understands its importance, given the fact that has a long history of solving big problems for very big companies.
But now the big solution providers have to not just have software that does things, but also need a methodology that goes with the software that helps customers be successful. To this end, Tibco is now investing in building up a partner channel of domain experts, says Rogers:
In order to do that you have to understand what the heck is happening in the industry, I think we do, and I think we’re really good at helping our customers get all the way down to what algorithm do you use, which is almost the final answer in that value. I’ll make a further point, which I think puts a bow on it. We’re at a point now where we’re helping customers figure out which algorithm to use, and we let the software tell them that. So they don’t necessarily have to know.
What is more, the people such businesses will turn to for help and advice will not be the mainstream vendors, but rather specialists with relevant domain expertise, the new re-intermediators. As Rogers observes, the underlying technology is now ready-made to support their growth, and the software vendors must have a methodology to support an encourage them. Start-ups can afford to do this now using open source tools, and can probably be viable within a week. So getting into enterprise data science and analytics can be done at a price point that isn’t nearly as prohibitive as it used to be.
Rogers acknowledged that the recent launch of Cloe by Densify was a good example of this new form of reintermediation in practice – where a domain specialist hits a specific sweet-spot business or operational need with AI and analytics.
Cloe applies analytical tools to one apparently simple task that is in fact stupendously complex – identifying appropriate cloud configurations for any given cloud workload in a world where the likes of Amazon AWS is estimated to be able to offer 1.7 million of them, a task where the data is well beyond being handled by any manual approach:
I think this is the route that we’re on. The life that you and I have been watching for quite some time is where most everything is manual, especially when it’s around data and around insight where humans were doing their best to make sense of the data. A lot of the automation that we’ve seen up until now has been business rule based, so it was still sort of human in a sense because a human has to write every single rule – if it’s red then we do this, if it’s blue then we do that.
Many of these new services will also be exploiting the cloud, for that is the obvious place for users to seek them out and start working with them. Rogers said he would feel a little stunned to meet a brand-new company that has an innovative service around analytics that’s not now starting out in the cloud and that a cloud-first culture is now the way to do things. This is not to say that an on premise or an inside-the-firewall private cloud implementation would not end up the best solution in the long term, however. One of the important decision points here would then be what he calls”the data gravity involved”.
The gravity of data
Some of this is determined by the type of data in question. Some is intrinsically better left behind the firewall on the source system. Typically this is any data that is sensitive in some nature, so users connect to it in managed (and probably authenticated) manner. Other data types can be allowed more freedom because there is less regulation and less risk surrounding its use. Here, users may well be allowed to collect it and put it into the cloud close to the target application.
This latter category, coupled with the cloud, is opening new opportunities for shared data that is of equal value to a class of stakeholder in, say a marketplace or operational area. For example, such open data is being shared by states, provinces and municipalities, either in its entirety right on the system, or by allowing users to connect to it under prescribed rules.
The other aspect of data gravity is its sheer size, he adds:
Data has a lot of gravity today, more so than it did when I got into this game, and the gravity tends to dictate the Living Space of the data. So if it weighs 400 petabytes, like the retailer Sears in the United States which has over 300 petabytes in production in a Hadoop cluster, then and it ain’t movin’! You’re not going to ship that to the cloud. So it’s there, it lives there, you’re going to connect to that data, you’re going to augment your intelligence up here in the cloud by connecting to that data.
The corollary of this is the growing number of opportunities to take the analytics to the data, running the analytics at the edge of an environment rather than bringing the data back to the centre. Rogers sees the Internet of Things as a really good example of data being generated that is continually building way out on the network edge.
But when it is being generated out there, and any monitoring and management actions need to be taken out there, it usually makes little sense to ship all that data back to the centre. Analyse it and utilise it where it needs to be used, urges Rogers:
This is what the new data landscape is looking like, and being able to offer both to users is becoming table stakes right now. If you can’t do it, I think customers are going to get pretty angry with the vendors.
Rogers’ thinking shows that at least one vendor is looking to ‘de-cerebralise’ as much of the practicalities of using IT as possible – AKA make the ‘T’ of IT as close to invisible as possible, so that users can then address the ‘I’ in as many different ways as seems relevant to them. How they manage the data, where they store and what algorithms they use to process it no longer become major decisions. Instead, high level guidance on the what and why of analytics will come from specialist partners with deep domain knowledge. This trend is perhaps another sign that IT is at last starting to grow up
Image credit - Stuart Lauchlan