Melanie Mitchell wrote a must read article for anyone trying to make sense of AI.
She objectively explains the state of AI as a scientist in an accessible way to the curious. These foundational AI chatbots launched in November 2022 often amaze us and frequently fail us in unhuman like ways. This inspired the new term: “jagged intelligence.”
While we are mostly in the experimentation stage with AI thinking (inference, chatbots), some people, teams, and enterprises are moving forward with the trial-and-error of AI doing (action, agents). Millions of AI agents with jagged intelligence released into the wild. Are we ready?
It is unlikely there is a better AI explainer than Melanie. She has written several books including Artificial Intelligence: A Guide for Thinking Humans.
Some of Melanie’s key observations:
Jagged Intelligence
A new term has been coined to describe AI in its current form: “jagged intelligence.” AI capabilities are profoundly uneven: the tools demonstrate excellent abilities on certain problems but surprising failures on other similar problems.
While AI boosters have touted superhuman capabilities of LLMs, other AI users have noticed their puzzling, unhuman like failures, which have not gone away as these systems have progressed.
How can a system, extensively trained to refuse dangerous requests, be easily fooled by “prompt engineering” into cheerfully providing the prohibited information?
Last fall, Ilya Sutskever, a cofounder of OpenAI, argued that there are no easy fixes to this problem: “These models somehow just generalize dramatically worse than people. It’s a very fundamental thing.”
Melanie provides an example of the unhuman like failures.
Potential sources of AI failures
Unlike LLMs we are embodied creatures with a sense of self, a sense of others, and (at least for most of us) a profound caring about the consequences of our actions. An LLM has no body, of course, no conception of itself as a “self,” and no self-generated desires or motivations.
Most in the AI field have treated these factors—embodiment, intrinsic drives, and engagement with the world—as irrelevant to intelligence and therefore to training machines to think
Jobs
In January of this year, Dario Amodei, the CEO of Anthropic, predicted that “we might be 6–12 months away from models doing all of what software engineers do end-to-end.” Sam Altman of OpenAI has said that by 2030, AI will replace 40 percent of human jobs.
There are (at least) two problems with such predictions. The first is that they are based on AI performance on benchmarks, which, as we’ve seen, has a poor record of predicting success in the real world. The second is that AI is tested on “tasks,” such as classifying medical images, writing computer code according to given specifications, or generating ad copy for real estate listings. But human jobs are not simply collections of independent fixed tasks; most jobs require the jobholder to understand how different tasks relate to one another, to adapt to change on the fly, and, more generally, to be flexible based on the open-ended nature of the real world.
For more from Melanie, check out her Substack AI: A Guide for Thinking Humans.