Enterprise hits and misses - smart cars aren't private, Dreamforce does AI, and ethical AI debt is a thing

Jon Reed Profile picture for user jreed September 18, 2023
Dreamforce is in the books - and we've got your curated analysis. AI took center stage - but how would I grade the approach? Smart cars might be smart, but they are also data privacy sieves. Real-time is a work in progress, and ethical AI debt is real. As always, your whiffs.


Lead story - Yes, ethical debt is a problem for AI software development

In one of my favorite enterprise AI pieces of the year, Neil warns about the problem of ethical debt.

Wake up call: AI projects bring the issue of ethical debt to a head, where the temptation to move fast comes at a potentially high cost. What is ethical debt? Neil:

Much like technical debt, where expedited solutions and shortcuts can lead to more significant problems down the line, ethical debt refers to the compromises made in ethical considerations during the developmental phases of AI technologies.

Rapid innovations might not sufficiently incorporate mechanisms to obtain informed consent from the users, thereby accumulating ethical debt by undermining the autonomy and privacy of the users. When the pace of development is frenetic, there might be inadequate engagement with a diverse range of stakeholders, leading to a lack of representation and inclusivity in technology development, which can foster ethical debt.  

Neil addresses the top factors that "lead otherwise reasonable people to pursue innovation at the expense of the 'good.'"  A couple of these jump out to me, starting with transparency:

AI technologies, especially deep learning models, can be incredibly complex and opaque. The fast pace of development can forgo the necessary steps to ensure transparency, making it difficult to ascertain how decisions are being made.

Neil details the ways ethical AI debt can accumulate. Needless to say, these are not easy issues to solve. Example: minimizing bias in AI systems adds an earnest touch to any keynote; accomplishing it is another matter. But ethical debt is a vivid way of framing AI project pitfalls, and I believe the debt analogy holds up: once you accumulate significant debt, getting out of it is not easily done.

Also see: Neil's Enterprise AI in practice - addressing the top CIO questions, where he takes on a number of provocative topics, including overhyped AI uses cases:

Until customer-facing systems can handle complicated, multi-step problems, "automated attendants" and DIY "My" sites will remain frustrating.

Diginomica picks - my top stories on diginomica this week

Vendor analysis, diginomica style. Here's my three top choices from our vendor coverage:

Dreamforce does AI (and data too)

Our wall-to-wall Dreamforce coverage is curated for your perusal. Watching from the east coast, I'll say this much: Benioff's keynote was one of the most direct, non-hyperbolic AI keynotes I've seen this year. That doesn't mean we escaped the AI hype festival; as Stuart documented on site, OpenAI's Sam Altman made the eyebrow-raising argument that AI hallucinations can be a good thing. But while I couldn't disagree with Altman more - particularly in an enterprise context where accurate decisions are everything - Altman is a major player and we need to track his arguments. I can't summarize all our Dreamforce news/analysis and use cases here, but this should get you started:

Jon's grab bag - three more AI drill downs stand out:

Finally, Brian Sommer and I wrote a (tough) love letter to the PR folks in our lives: How did your tech news story wind up in the circular file? The top ten ways PR pitches go wrong. Which brings to mind:

Best of the enterprise web

Waiter suggesting a bottle of wine to a customer

My top five


Overworked businessman


I'm gonna wrap these whiffs quickly, or the real whiff will be on me for losing this column on bad airport wifi.

Couldn't resist this gratuitous double shot at Google Drive (and LinkedIn braggadocio):

Speaking of gratuitous shots, I couldn't resist this one either:

I know there are some good AI use cases out there, under a rock at a glitzy hotel in Vegas may well perhaps... Given my upcoming travel schedule, I'll find out soon.

If you find an #ensw piece that qualifies for hits and misses - in a good or bad way - let me know in the comments as Clive (almost) always does. Most Enterprise hits and misses articles are selected from my curated @jonerpnewsfeed.

A grey colored placeholder image