Enterprise hits and misses - smart cars aren't private, Dreamforce does AI, and ethical AI debt is a thing

Lead story - Yes, ethical debt is a problem for AI software development

In one of my favorite enterprise AI pieces of the year, Neil warns about the problem of ethical debt.

Wake up call: AI projects bring the issue of ethical debt to a head, where the temptation to move fast comes at a potentially high cost. What is ethical debt? Neil:

Much like technical debt, where expedited solutions and shortcuts can lead to more significant problems down the line, ethical debt refers to the compromises made in ethical considerations during the developmental phases of AI technologies.

Rapid innovations might not sufficiently incorporate mechanisms to obtain informed consent from the users, thereby accumulating ethical debt by undermining the autonomy and privacy of the users. When the pace of development is frenetic, there might be inadequate engagement with a diverse range of stakeholders, leading to a lack of representation and inclusivity in technology development, which can foster ethical debt.  

Neil addresses the top factors that "lead otherwise reasonable people to pursue innovation at the expense of the 'good.'"  A couple of these jump out to me, starting with transparency:

AI technologies, especially deep learning models, can be incredibly complex and opaque. The fast pace of development can forgo the necessary steps to ensure transparency, making it difficult to ascertain how decisions are being made.

Neil details the ways ethical AI debt can accumulate. Needless to say, these are not easy issues to solve. Example: minimizing bias in AI systems adds an earnest touch to any keynote; accomplishing it is another matter. But ethical debt is a vivid way of framing AI project pitfalls, and I believe the debt analogy holds up: once you accumulate significant debt, getting out of it is not easily done.

Also see: Neil's Enterprise AI in practice - addressing the top CIO questions, where he takes on a number of provocative topics, including overhyped AI uses cases:

Until customer-facing systems can handle complicated, multi-step problems, "automated attendants" and DIY "My" sites will remain frustrating.

Diginomica picks - my top stories on diginomica this week

• Dun & Bradstreet - accurate data must be the basis for any serious enterprise use of generative AI - Gary bears down on what enterprise AI will take, and how Dun & Bradstreet is commercializing its AI learnings.
• Grieves and Vickers - digital twin challenges and opportunities - George builds on his digital twins series, with a generative AI twist. Interesting NASA story: "As it turned out, the digital twin predicted the point of failure to within three percent and for a few tens of thousands of dollars rather than hundreds of millions for the physical prototype. However, the existing culture mandated spending hundreds of millions of dollars on the physical prototype."
• AI regulation - Stability AI and Ada Lovelace Institute speak out - Chris delves further into the problems of AI compliance, via his three part mini-series: "The reality, I suspect, is that some vendors – we won’t name them – are fully aware that they scraped private and/or copyrighted data to train their systems. But they can’t admit to it as it would generate a flood of lawsuits."

Vendor analysis, diginomica style. Here's my three top choices from our vendor coverage:

• The "deep water of the Information Age" offers Oracle great opportunity, argues CTO Larry Ellison, but Wall Street takes a cool view of Q1 numbers - with Oracle CloudWorld on deck this week, Ellison navigates the fickle corridors of Wall Street. Stuart also makes sense of this frenemies-type announcement, Why Oracle is putting OCI database services into Microsoft Azure data centers. (Derek, Brian and I are in transit to a week of Oracle CloudWorld hijinks as I type).
• Crater Labs fuels demanding AI projects with Pure Storage - Derek has an interesting use case at the intersection of cutting edge data storage and LLMs.
• Celonis’ new EMEA chief focused on time to value in tough economic times - a progress report from Derek. This quote from Celonis jumps out: "It's not about spending more, there is nothing more to spend. It’s reprioritizing those and prioritizing those that have a 10X return or more."
• The pace of change is the slowest it will ever be - how Genpact is using AI for digital transformation - Alex examines the skills imperative brought on by AI. Pull quote from Genpact: "The talent that you hire today doesn't really have the skills of the future."

Dreamforce does AI (and data too)

Our wall-to-wall Dreamforce coverage is curated for your perusal. Watching from the east coast, I'll say this much: Benioff's keynote was one of the most direct, non-hyperbolic AI keynotes I've seen this year. That doesn't mean we escaped the AI hype festival; as Stuart documented on site, OpenAI's Sam Altman made the eyebrow-raising argument that AI hallucinations can be a good thing. But while I couldn't disagree with Altman more - particularly in an enterprise context where accurate decisions are everything - Altman is a major player and we need to track his arguments. I can't summarize all our Dreamforce news/analysis and use cases here, but this should get you started:

• Dreamforce 2023 - “We call them hallucinations; I call them lies!” Benioff’s call to arms around the generative AI trust gap (Stuart). Also see: Dreamforce 2023 - piloting Einstein Copilot in the user experience of every Salesforce application...and you only have to ask.
• Dreamforce 2023 – Lamborghini streamlines multiple apps and disparate data as Salesforce tech stacks up (Madeline)
• Dreamforce 2023 - Salesforce harnesses metadata for its new Einstein 1 platform based on Data Cloud (Phil) - and Phil's follow-on: Four reasons why the new Data Cloud is so strategic for Salesforce
• Dreamforce 2023 - the wrap (Stuart) (p.s. one of the lingering post-Dreamforce stories is the future of Dreamforce in San Francisco, but as per Stuart's Dreamforce 2023 - why the AI boom is an opportunity for everyone, according to Salesforce CEO Marc Benioff, it looks like some clues of a San Francisco Dreamforce future are emerging.)

Jon's grab bag - three more AI drill downs stand out:

• Taking a careful approach to AI in marketing - Barb
• How AI impacts the publishing sector, and what needs to be done to protect that industry - Chris
• AI is ‘letting lawyers lawyer’ at Clyde & Co - Gary

Finally, Brian Sommer and I wrote a (tough) love letter to the PR folks in our lives: How did your tech news story wind up in the circular file? The top ten ways PR pitches go wrong. Which brings to mind:

Best of the enterprise web

My top five

• 8 Keys to Managing the Linguistic Copycats that are Large Language Models – Hyoun Park of Amalgam Insights adds to his strong series of generative AI posts. This one includes pointers on crafting more effective prompts: "From a practical perspective, this means that the more context and expertise provided in asking an LLM for information and expected outputs, the better the answer that will typically be provided."
• Businesses need pricing clarity as generative AI services hit the market - timely post by Eileen Yu at ZDNet: "Transparency around how exactly services are charged will be essential as organizations look to avoid bill shock." Indeed - I wrote about this on diginomica today; more to follow next week.
• The real-time revolution is here, but it's unevenly distributed  - another (essential) techno-buzzword cold shower from Joe McKendrick: real-time is a work in progress. Why? Oh, you know, the usual: technical barriers, real-time is really expensive, it's not always necessary, and so on.
• Demand Planning: Whipped And Chained by Tradition –Lora Cecere nabs the "tell us how you really feel award," and not for the first (or last) time: "An Outside-in Model Beats an Inside-Out Model Every Time. I laugh when I watch the current APS demand planning technologists scrap it out at a manufacturing site, with each trying to prove they have a better optimizer. In each case, the change in the optimizer from one technology to another had little impact on the inside-out model."
• It’s Official: Cars Are the Worst Product Category We Have Ever Reviewed for Privacy - ouch! "The gist is: [car companies] collect super intimate information about you -- from your medical information, your genetic information, to your 'sex life' (seriously), to how fast you drive, where you drive, and what songs you play in your car -- in huge quantities."
• Podcast bonus - if you're looking to liven your commute with jugular enterprise content, I've got a couple fun ones for you, from my Busting the Omnichannel show: The SAP innovation debate - hashing out SAP’s RISE and AI plans with Greenbaum and Scott. And: the audio version of my new monthly show with diginomica contributor Brian "Brain" Sommer: The enterprise month in review - show debut with Brian Sommer.

 

Whiffs

I'm gonna wrap these whiffs quickly, or the real whiff will be on me for losing this column on bad airport wifi.

Couldn't resist this gratuitous double shot at Google Drive (and LinkedIn braggadocio):

Speaking of gratuitous shots, I couldn't resist this one either:

I know there are some good AI use cases out there, under a rock at a glitzy hotel in Vegas may well perhaps... Given my upcoming travel schedule, I'll find out soon.

If you find an #ensw piece that qualifies for hits and misses - in a good or bad way - let me know in the comments as Clive (almost) always does. Most Enterprise hits and misses articles are selected from my curated @jonerpnewsfeed.