The perils of the AI predictions game - lessons from 2021's AI predictions

Profile picture for user Neil Raden By Neil Raden December 29, 2021
Predictions are a perilous game indeed. But as we head into what is sure to be an AI-saturated 2022, a look back at these ten AI predictions from 2021 is instructive.

Hand ringing service bell held by robot © Andrey_Popov - shutterstock

Niels Bohr, the Nobel laureate in Physics and father of the atomic model, is quoted as saying, "Prediction is very difficult, especially if it's about the future!" I picked Ron Toews, not to criticize, but to use his AI predictions for 2021 (made last year) as an example of how harrowing this game can be.

I'll admit, I'm too timid to make predictions, but I do enjoy tracking how well others did in the previous year. Also, Toews is very well informed about the subject matter, and he's not a vendor, whose "predictions" are not always objective.  

Here's my quick hits on Toews' ten AI predictions for 2021 - and how they fared. For context, I would encourage you to read Toews' full article, 10 AI Predictions For 2021.

1. Both Waymo and Cruise will debut on the public markets.

As a rationale, Toews noted that:

The 2020 SPAC boom has provided a novel way for less mature businesses to go public. And SPAC investors have shown a voracious appetite for next-generation mobility companies.

(e.g., Nikola, Velodyne, Luminar, InnovizCanoo, FiskerRomeo Systems).

My review: Waymo was acquired by Alphabet. In March 2021, Cruise acquired Voyage, but in 2016, Cruise was owned mainly by GM, which did not spin Cruise off in 2021. Side note: Nikola was already a publicly-traded company in 2020.

2.  A political deepfake will go mainstream in the U.S., fueling widespread confusion and misinformation.

My review: Lawmakers and researchers had warned that videos altered using AI could disrupt the 2020 vote. But they didn't turn out to be a problem. See: What Happened to the Deepfake Threat to the Election?.

3. The total number of academic research papers published on federated learning will surge.

My review: This is the type of prediction that isn't a prediction. What does it mean "to surge?" The most comprehensive peer-reviewed article on federated learning is Federated Learning: A Survey on Enabling Technologies, Protocols, and Application, and it has many citations about the subject, but I don't see very many from 2021. I wrote about federated learning here: Federated Learning, Differential Privacy: Healthcare Proceeds one Funeral at a Time. However, based on his data, Toews does believe his "surge" prediction was correct.

4. One of the leading AI chip startups will be acquired by a major semiconductor company for over $2B.

My review: This one was way off: AI chip startups achieved notable funding rounds, but none were acquired near that price range:

We knew the day would come when an AI chip startup hit the $1 billion funding mark, and so it has. SambaNova Systems has announced its Series D round of a fresh $676 million with SoftBank leading the charge, bringing the company to over one billion, with a valuation that SambaNova says is at the $5 billion level.

Sidenote: Toews acknowledged this one was wrong, noting that: "Instead, the leading AI chip startups all raised rounds at multi-billion-dollar valuations, making clear that they aspire not to get acquired but to become large standalone public companies."

5. One of the leading AI drug discovery startups will be acquired by a major pharmaceutical company for over $2B.

My review: This isn't really much of a prediction - several of these firms ended up going the IPO route instead.

6. The U.S. federal government will make AI a true policy priority for the first time.

My review: This is true, including significant movement in Congress, DoD, NSAC, JAIK, even the FTC. I wouldn't go so far as to say any of these are actually "policy," but they are moving in that direction. See my piece: What is the role of AI in pandemic response? The National Security Commission on AI provides a framework.

7. An NLP model with over one trillion parameters will be built.

My review: OpenAI's GPT-3 had 175 billion variables, and it was already working on GPT-4, but concerns were raised that the enormous cost of computing  (more than $5M in electricity) was too much. OpenAI now claims they are building GPT-4 with a more economical computing model.

However, as Toews noted, Google did indeed build such a model in 2021: "In January 2021, less than a month after we published our predictions, Google announced that it had trained a model with 1.6 trillion parameters, making it the largest AI model ever built."

8. The "MLOps" category will begin to undergo significant market consolidation.

My review: "Begin to" are the kinds of predictions that aren't predictions. However, the list of potential targets is interesting, but no one from this list was acquired:

Alectio, Arize AI, Arthur AI, Comet, DarwinAI, Fiddler Labs, Gradio, OctoML, Paperspace, Snorkel AI, Truera, Verta, Weights & Biases

Potential acquirers IBM, Microsoft, Amazon, Databricks, DataRobot, Domino Data Lab, and Oracle did not purchase them.

Nevertheless, some MLOps companies were indeed acquired. Toews: "Probably the most noteworthy example came in July with DataRobot’s acquisition of Algorithmia, which had raised close to $40 million in venture capital funding. Other examples include HPE’s acquisition of Determined AI and DataRobot’s acquisition of"

 9. AI will become an important part of the narrative in regulators' antitrust efforts against big tech companies.

My review: Again, "will become" is too squishy to be a prediction. Toews believes he got this prediction right, but it's not certain the DOJ will use antitrust to moderate the big tech companies, which make up over $2T of the GDP. However, at the end of 2020, this was already an issue, so it's not a prediction.

10. Biology will continue to gain momentum as the hottest, most transformative area to apply machine learning. 

My review: Again, phrases like "will continue" are not predictive. Besides, I'm unsure what the "hottest, most transformative area" is. If you believe Yuval Harari, biology, biometric information, and devices will destroy humanity within one year.

End note: as indicated, Toews has published a detailed self-review of his 2021 AI predictions. He has also issued his AI predictions for 2022.