-
AI faces several long-term issues, including degeneration
-
These come with maintenance costs enterprises will need to plan for
-
The best way forward in the short term is to take a cautious approach to AI implementation
For humans, aging is scary. That’s not just because our skin starts to sag here and there. For many, aging comes with mental impairments as well, including Alzheimer’s, dementia and a million other conditions that we can’t even name. My own mother, for instance, had a degenerative neurological condition that caused her to slowly lose control of her motor functions. Turns out something similar could happen to artificial intelligence (AI) models if we’re not careful.
AI “dementia…is something that we’ll start to discover becoming the big problem that a lot of these AIOps lifecycles are going to have to address. But more importantly, it’s going to be a cost that any enterprise needs to invest in,” Leonard Lee, founder of analyst firm next Curve, told Silverlinings.
“It factors into the cost of implementing these. So, anyone who has the notion that this is going to be a productivity uplift tool, needs to factor this into the overall ROI equation,” he added.
That means accounting for testing and monitoring as well as updates, he said. Woof. As if battling cloud costs wasn’t enough for enterprises to deal with.
Lee first brought up the subject of AI dementia (a.k.a. AI degeneration) at Silverlinings’ Cloud Executive Summit in December. But what exactly does he mean when he uses the term?
Watered-down AI
When we reconnected with him this week, Lee explained that degeneration is separate from hallucination and even model collapse, a wild degenerative defect that can occur when models ingest AI-generated data during training. (And man is that messy. Really, read the paper in the link. It’s nuts.)
What he is talking about is basically dilution. As models age, they are trained and retrained with more and more and more input parameters. Even setting aside the dirty data problem, that process effectively dilutes the model’s ability to return domain specific answers and can make the model less effective in producing accurate or valuable outputs.
Basically, the model ends up with too much generalized knowledge to the point that it can’t dig deep enough on any one subject.
Lee said some are exploring ways to overcome this by ditching generalized models and developing what is known as a “mixture of experts.” Put simply, the latter is one big model comprised of smaller, domain specific expert models that battle it out internally to produce the most accurate answer. The idea is that by limiting the number of input parameters in the expert models, the overall model will retain its accuracy.
But Lee said the matter is far from settled. Though he said he’s taken the question to experts Nvidia, Amazon Web Services and the like, no one has a definitive answer yet for the problem of cognitive decline.
Limitation boogie
Lee noted cognitive decline is just one of several limitations AI models face. And he said enterprises should listen carefully when AI leaders like OpenAI warn them about the flaws that still exist in their models.
As Andrej Karpathy, OpenAI AI researcher, put it during Microsoft Build in May 2023: “Models may be biased. They may fabricate, hallucinate information. They may have reasoning errors. They may struggle on entire classes of applications. They have knowledge cutoffs, so they might not know any information about, say, September 2021. They are susceptible to a large range of attacks, which are sort of like coming out on Twitter daily, including prompt injection, jailbreak attacks, data poisoning attacks and so on.”
His advice for enterprises while all this gets sorted out? Use AI for “low-stakes applications” and always keep a human in the loop.
This article was updated on Monday, Feb. 5, 2024 at 5:58pm EST.