Ethan Mollick has released a new post on his blog, One Useful Thing, on Substack, and it’s not what you’d expect, really.

I always follow posts from this prominent voice on AI; with Mollick’s MIT connections, and his record of being relevant on the evolution of LLMs, I’ve been watching him analyze digital otters and such for a number of years.

The newest piece references a “cognitive overhang,” pointing out that in some ways, the systems of the future that we imagine are the systems that already exist today.

“While there is a lot of debate over what future models will do, the current capabilities of existing models are barely being used, and are often not even well understood,” Mollick writes.

That’s a hard one to get your head around, but he does provide some context.

But there’s also, in Mollick’s new demos, a rather large easter egg, in terms of his new book. In a meta rabbit hole twist, the post includes an AI video hawking the book, Coexistence, which comes out October 20. Just in time for Halloween. Which is, in a way, kind of what it feels like to watch the results of Mollick’s prompting cosplay an interaction between a human and an AI, giving the idea of recursive generation a brand new face.

In the demo video, which is only about half a minute long, a helmeted and otherwise anonymous human figure rides a motorcycle away from apparently hostile aircraft, billowing apocalyptic flame, and something that looks rather like what an undergrad would build with C++.

“Tell me you have a plan,” the apparent human user, with a female-sounding voice, demands.

“12,000.” returns the apparent AI laconically, in a male-sounding voice with a strange Anglo-Australian type of accent.

“Pick one!” the user shouts, in aggravated tones.

Then there’s this exchange: AI, human, AI.

As the motorcycle hums along a rooftop, the user then asks the companion entity this question:

“Did you check the landing?”

“I thought you were doing that,” the AI said.

Mollick describes how an LLM made this for him.

“It … figured out how to generate voices and music and sound effects, and gave me this film 45 minutes later,” he writes. “The final product feels a little more ominous than I would like, but that was the AI’s decision, not mine.”

The implication seems to be that the job of the human in the loop needs to check the work of the AI, or more specifically, make sure that the AI has addressed all of the right parts of a task or plan. But Mollick breaks this down further in commentary, also suggesting that humans have a quartet of advantages over their digital peers, having to do with the idea of curation as opposed to endless generation. I’m reminded of the first time I researched a generative adversarial model, which actually consists of a pair of models: one to generate, and the other to discriminate. In Mollick’s suggestion, this second part should be in the human domain, although you have to ask yourself: how much longer until AI is doing both?

In this simulated example, if the AI didn’t check the landing, and the human didn’t check whether AI checked the landing, the unchecked landing could tether a neat getaway to a denouement of flame and peril. There’s the sort of tongue-in-cheek throwback to a pair of humans arguing about who shut the door, or didn’t shut the door, or whatever. But the point is well taken. Don’t just let AI do stuff.

The four advantages that Mollick enumerates after showing us the video are as follows: deep knowledge, wide knowledge, taste, and agency.

“The first two advantages come from what you know,” he writes. “Deep knowledge is the expertise that comes from understanding a field or subject so well that you build intuition around it to quickly and accurately make decisions. It is how an experienced accountant can glance at a spreadsheet and know something is wrong, or how a golf pro can watch a swing and instantly understand the mistake the golfer is making.”

Then there’s the wide knowledge, of which Mollick writes:

“The training data for LLMs is a large swath of humanity’s vast output. The AI has learned something of design thinking, and Bayesian reasoning, and the Toyota Production System, and Rogerian therapy, and Marxist literary criticism. But AI tends not to volunteer any of these patterns unless you know to ask. This is where wide knowledge comes in.”

The result is evident, he notes, in how AI handles design work.

“If you ever ask AI to create a webpage, it will have certain preferences, including a very annoying habit of adding little headlines on top of your headlines,” Mollick writes. “If you don’t have any grounding in design, you may not realize that you need to ask the AI to stop ‘adding eyebrows’ to the work… to gain wide knowledge, you need to read and study widely, across fields and formats and traditions. This is valuable in and of itself (the return of the liberal arts!) but doubly so in the age of AI.”

I like the call-out to the humanities, which we seem to be pushing onto certain ideological reservations, in what I think a lot of us would agree is a short-sighted form of monomania. Won’t we want this stuff back later?

The goal is curation. Mollick has this to say about human taste:

“Generative AI leads mostly to slop: a flood of work that is very similar to each other. But slop can be defeated by taste. Making great things with AI means knowing which AI outputs to keep, which to discard, and which to use as raw material for something the AI would never have generated on its own.”

“The jagged frontier is unmapped in your field, so agency is about becoming an explorer,” he writes. “It’s the difference between waiting for someone to tell you that AI can now do something, and discovering it yourself by trying.”

There are your four north stars. So think about this when you’re starting your next project – and look for that book next month.