Lead story - on AI hype, field lessons and hallucinations
As we enter the final stretch of 2023, the pros and cons of enterprise AI remain a preoccupation, and rightly so. AI success (and ROI) is not pre-ordained; there is a difference between a flawed and a successful approach.
Exactly what the difference is? Well, that's one of our big jobs to refine in 2024.
Start with George's AI lessons from financial market surveillance, which brings the question of AI regulations into focus. Enterprises that are embarking on AI projects need to include the prospect of stronger regulations as part of their risk assessment. The finance industry is already grappling with this; George quotes Deloitte's Roy Ben-Hur:
As AI transitions to support more decision-making in the surveillance space, we are expecting regulatory expectations to increase, requiring firms to provide greater transparency and clarity on items such as how model data is being sourced and used, how potential biases are identified and managed, the logic that describes algorithms, and comparisons between established policies, and detection of potential tampering.
In TruEra - baking hallucinations into AI quality assurance, George takes on another generative AI adoption barrier: the problem of hallucinations. Some approaches to reducing hallucationations are one-offs; but TruEra is pushing for a more comprehensive approach:
At the moment, researchers and vendors are all struggling to find the most efficient way to measure and reduce hallucinations in AI. This will be critical for new generative AI to scale in the enterprise. More importantly, some of the best hallucination metrics can be a one-off technique. TruEra is approaching the problem as part of a broader solution to streamlining the AI development lifecycle.
As enterprise projects press ahead, they are keeping an eye on the AI downside - and another wary eye on vendor (over)hype. Mark Samuels documents this in How Walgreens Boots Alliance avoids AI vendor hype, where the analogy to RFID hype is a bit of a cold shower. He quotes Walgreen's CIO:
We're very deliberate in what we're using across our businesses – and the use cases are not the same. What we're using in pharmaceutical wholesale is completely different than what we're doing in retail. But it's all about using AI and data in a proportional way because you can get skewed and get caught up in the hype.
Another huge customer theme? AI and date platform readiness. Sometimes the smartest initial AI project is not an AI project at all, but a postponed data initiative. An upgrade in security approaches, which one customer at BMC Connect referred to as "security by design," is also an imperative. Mark gets into this in Get your IT foundations in place before you start exploring AI, quoting Lisa Murray Brown from the Ministry of Defence:
It’s about getting the IT basics and the data right. It’s about focusing on what the minimum viable product is in order to move forwards. Then, it’s about establishing what your priorities are. Get the basics right before you do all this sexy stuff.
The tensions between AI potential and due diligence should keep us busy in 2024.
Diginomica picks - my top stories on diginomica this week
- CIO Interview - merger provides Sovereign Network Group with new working methods opportunity - Mark Chillingworth filed a couple of meaty CIO discussions. One big takeaway: time to rethink field work as much as we have done the same for office-based work. Mark quotes Sovereign Network Group's CIO: "We have already done a lot of work on the hand-over processes of a home, so it is much slicker. This means that when a customer calls the maintenance team, then that team has the data available to them. But there is a lot of untapped potential – if we can get better at capturing real-time data about our homes then we will be able to anticipate issues with them before they become a problem for our customer – for example, the hardware we are rolling out that detects moisture levels which could lead to damp in a home." Also see: Mark's CIO interview - Deutsche Bahn builds central IT platform for change.
- Citizen Developer - your time has come! - Has AI and low-code converged to enable the true emergence of citizen developers? Martin explores the possibilities.
Vendor analysis, diginomica style. Here's my three top choices from our vendor coverage:
- Solid Q3 2024 for MongoDB as it anticipates future growth from AI tools and application modernization - Derek on MongoDG's strong outing, which centers around AI app building and customer platform modernization: "Customers are talking to MongoDB about the pressure that they feel to modernize their data infrastructure, as they wake up to the realization that their legacy platforms will face limitations within the context of what could be an AI-driven economy."
- Box share price dips as third quarter earnings disappoint - After a series of strong enterprise earnings reports on diginomica, Box struggles. Why? As Derek reports, Box thinks that an AI-powered approach to content will help them emerge stronger: "CEO Aaron Levie told analysts during the company’s earnings call that Box is focused on driving profitable growth at scale - and that in FY ‘25 the focus will be on driving investments across AI, security, advanced content management and workflow capabilities."
A few more vendor picks, without the quotables:
- Salesforce and AWS - why their marketplace alliance matters - Rebecca
- Planful CEO Grant Halloran - how collaborative, continuous planning can change the way your business operates - Phil
- Certinia under new leadership - an exclusive interview with DJ Paoni, its new CEO - Phil
Jon's grab bag - Chris continues his examination of AI's impact on industry, this time with a legal focus: AI in legal services - will lawyers or citizens win in this battle for the future? But who won this round? The AI evangelists or the critical thinkers? "The real danger is not the technologies themselves, but those who rush to adopt them uncritically. And, sometimes, those who ignore them until it is too late." See also: Chris' How will generative AI impact legal services? It’s all about responsibility, say lawyers.
Don't fall for data governance bromides, wants Neil in his scorching Want to solve data governance failures? First, move past nonsensical advice. Finally, Brian and I issued our tonic for tech-predictions-inbox-syndrome, The 2024 enterprise software un-predictions. Our new tech buzzwords sparked discussion, such as the "data outhouse," "crapform" and "hydromatic automation." You can build your own un-predictions with our un-predictions generator - let us know what you come up with...
Best of the enterprise web
My top seven
- Re-founding, reInventing and the Future of AWS - I wasn't able to surface a strong piece of analysis on AWS re:Invent last week, but this week we have RedMonk's Stephen O'Grady, delving in as AWS looks to make up for missed ground in the AI push.
- Retail and CPG firms turn to innovation to combat the digital dichotomy - Phil Fersht and the HfS team filed an incisive view of retailers at the innovation crossroads, with tight margins on the one hand, and enticing-but-yet-unproven gen AI solutions on the other.
- Supply Chain Resilience. Really? – to Lora Cecere, supply chain resilience is an oxymoron more often than not. Here, she explains why - and what she will say/has said in recent speaking appearances.
- C3.ai's next move: Convert generative AI pilots to production deals - Constellation's Larry Dignan with a useful analysis for C3.ai's ups and dows. C3.ai is an enterprise AI vendor worth watching, but it's not all smooth sailing: gen AI pilots don't automatically translate to bigger projects.
- What's gonna happen with generative AI and CX in 2024? - Thomas Wieberneit with a CX angle on what's next for gen AI. Like Wieberneit, I am interested to see the economics - and value - of smaller, more specialized LLMs.
- Overcoming Common Challenges in ERP Implementation - Eric Kimberling is back in his wheelhouse with a thorough overview of the factors that define ERP project success (or failure).
- The future of SAP tech events - and the great SAP innovation debate - Josh Greenbaum and ASUG CEO Geoff Scott rejoin me for a conversation that went off the rails, in the best way possible.
A couple of gen AI whiffs - though the decline of once-venerable publications like Sports Illustrated isn't AI's fault. Still, this isn't a good look either:
The depressing fall of Sports Illustrated reveals the real tragedy of AI https://t.co/3GvwDC2JW0
"notably publishing an article whose sole purpose was to list the “Star Wars” movies in order and yet got the order wrong — it caused an uproar."
-> watch them in any order lolz
— Jon Reed (@jonerp) December 10, 2023
More ChatGPT whiffery:
Via 404 media, an awkward situation indeed: Male Tech Conference Founder Is Behind Popular Woman Coding Influencer Account: "IP logs show that accounts for Coding Unicorn, a female tech influencer who's built a following based on her coding advice and Instagram influencer posts, are run by a male developer and conference organizer." Another one for the year's 'This one is going to end badly" file... See you next time.
If you find an #ensw piece that qualifies for hits and misses - in a good or bad way - let me know in the comments as Clive (almost) always does. Most Enterprise hits and misses articles are selected from my curated @jonerpnewsfeed.