From Artificiality This Week * Our Gathering: Our Artificiality Summit 2025 will be held on October 23-25 in Bend, Oregon. The
AI agents are evolving with orchestration layers enabling modular, autonomous systems beyond traditional SaaS, reshaping workflows, labor, and software design.
David Wolpert warns AI networks may evolve beyond human math, creating unpredictable, emergent intelligence that defies control and comprehension.
ChatGPT Will Never Work Like the Demo, AGI is a Red Herring, The Brittleness of Agentic Reasoning and Planning Using LLMs, Can LLMs Reason and Plan, How to Use Generative AI to Create, and more!
In the New York Times, Brian X. Chen provides a solid comparison between the current state of ChatGPT-4o and that which was demoed at the launch. He claims “the demo turned out to be essentially a bait and switch.” He describes and shows in videos how the features that were most celebrated in the launch event are not replicable today. I wholeheartedly agree with his review—but want to take it one step further.
As a bit of background, tech demos are always a show with varying degrees of reality. It’s become accepted that companies will demo products and features that aren’t available yet. And it’s become accepted that demos will showcase only the best features—and likely dance around weaknesses. I understand this process well—I used to script and deliver demos for dozens of software products. The aim is to excite a customer with the best you have to offer while staying true to reality.
That said, there are two differences between demos as we’ve come to understand them and those that seem to becoming a pattern for generative AI products.
First, as Brian demonstrates, OpenAI released ChatGPT-4o in a state that does not match the announcement or demo. It’s one thing for Apple to announce the next iOS before it’s available. It’s another for OpenAI to release ChatGPT-4o in an incomplete state. The public has a justified mental model that the available version of a product is the product. If I choose to use ChatGPT-4o, it should have the same features as were marketed and the same features as everyone else.
But OpenAI is building a pattern of releasing a product which is actually just a beta version of the product. ChatGPT-4o is not yet finished. It is incomplete. And while OpenAI’s core fanbase may be fine with being beta testers, the general public isn’t and shouldn’t be.
This release strategy leaves people confused. When will ChatGPT-4o be feature complete? When will the voices from the demo be available? When will the desktop app be able to screen grab as in the demo? When will ChatGPT-4o actually be ChatGPT-4o?
Second, generative AI products will never replicate demos—ever. With traditional software, you can perform a demo and know with 100% certainty that a user will be able to replicate the demo. With generative AI, you can perform a demo and know with 100% certainty that a user will never be able to replicate the demo. This is the fundamental nature of predictive technologies—and I can’t stress enough how important this difference is.
If you ask ChatGPT the answer to physics problem, it isn’t looking up the answer. It isn’t querying a database of ground truth. It isn’t checking the answer key to the physics AP test. It doesn’t “know” any facts in the way that fits the mental model of human knowledge.
ChatGPT (and all generative AI language tools) is “reading” your prompt and then iteratively predicting the next “almost” best words to provide a response. I say “almost” because large language models have been tuned to predict words that are considered almost next best to create the kind of language variability that sounds human-like. This means that every time—every time—you provide your prompt, you will get a different answer.
This also means that every time—every time—you try to replicate a demo of a generative AI tool, you will get a different response than the demo. These unpredictable generations are core to why generative AI is so powerful and compelling. But they do not match our mental models for computers and are creating mass confusion.
One of the best lessons I learned from Steve Jobs was when, during a weekly software review, he turned around from the software he was testing and said, “Do your customers understand how this works?” I think it’s fair to say that for the majority of ChatGPT users, that answer is “not well enough.”
It’s time for OpenAI to redesign the entire user experience with a beginner’s mind so that people know what they’re really working with.
The AI Mirror, How to Reclaim Our Humanity in an Age of Machine Thinking, by Shannon Vallor
Vallor's book on AI and humans is the best of its kind I've read in years. Her writing style is a perfect fit for how I absorb information. Crystal clear and jargon-free, her descriptions of the tech itself are spot-on yet totally accessible.
Vallor is the Baillie Gifford Chair of the Ethics of Data and AI at the University of Edinburgh and has worked with AI researchers and developers for many years. She describes herself as a virtue ethicist and evaluates AI's reflection of us by considering how it may alter our perspectives on virtues versus vices. She embraces, elaborates on, and wrings every drop out of the metaphor of AI as a mirror.
This starts with how AI is built on a foundation of historical data, which means that humanity can't afford to rely on it. If we do, we risk dooming ourselves to being trapped in the past. "The conservative nature of AI mirrors not only pushes our past failures into our present and future; it makes these tools brittle and prone to rare but potentially spectacular failures," she writes. Touché.
Many otherwise familiar ideas gain new depth through her interpretations and scholarship. I discovered numerous concepts she brings to light from the history of technology philosophy. For instance, the notion that AI mirrors make us more like machines (instead of making machines more like humans) was termed "reverse adaptation" by the philosopher of technology Langdon Winner in 1977. Today, we see this with workplace surveillance transforming workers into metric-monitored automatons of efficiency.
Perhaps what I appreciated most in this book was her scathing appraisal of AGI, much of which I completely agree with. There are so many brilliant sentences! One that particularly stands out encapsulates the emerging anxiety about AGI (despite Sam Altman's enthusiasm for the idea) by drawing a parallel to the factories of the nineteenth century: "Visions of AGI overlords that cruelly turn the tables on their former masters mirror the same zero-sum games of petty dominance and retribution that today drain our own lives of joy."
But it's not all doom and gloom. Vallor highlights instances of AI being used as a tool for cultural recovery within Indigenous cultures, serving as a mechanism for reclamation. Another example is "reverse bias," where AI helps doctors become more aware of the historical under-treatment of Black people's pain. These are small but significant glimmers of hope. They highlight one of the values of AI: by revealing and measuring such issues, we can learn to reason about them differently.
The AI Mirror is worth your time if you're looking for a realistically skeptical view of tech with glimmers of hope for a more virtuous future with AI.
The Artificiality Weekend Briefing: About AI, Not Written by AI
