For those who have been following along with the AI story since the stuff was new, we are in a very specific moment right now, where what some investors might call FUD (Fear, Uncertainty, and Doubt) is higher than it’s ever been, in some ways. Let me explain.

It seems like for the very first time, now, there’s a real sense that this could go awry. It probably has to do with researchers and engineers for the big companies jumping ship and providing their testimonies to the press – although that has happened, to some extent, before.

But think back to any of those moments, through the past couple of years, as machine learning became deep learning, and deep learning became …. Something else. AI-written poetry? AI-generated graphics? AI writing code? Cool! (Very few people moaned for the writers and musicians and artists – and many of them were the writers and musicians and artists.)

Now. However, the humans are looking at real agentic collaboration, culminating in things like the Hugging Face incident, and many are a little spooked.

So I thought this was a very good conversation at the Imagination in Action The Next Endeavor event, in Mountain View, CA, where a panel discussed those big questions around whether we can really control AI, or not. (Disclaimer: With Imagination in Action, I help to put on these events.)

I thought that moderator Bradley Olson started off appropriately with the observation that you can’t have a recorded conversation in San Francisco (or, increasingly, anywhere else) without turning on the AI note-taking system, where the technology builds a detailed profile of everything you say, and maybe even how you sit.

That in hand, he asked the panel to think about how much of the current problem has to do with a simple lack of controls, and not a fundamental, permanent threat of rogue AI. In other words, with a simple set of protocols, can we contain these agents?

Panelist Noelle Russell had a prop: a little stuffed tiger she had brought with her from home.

“AI is kind of like a baby tiger,” she said. “Super cute. We all love it. We’re like, ‘Oh my gosh, I can’t wait to show my friends. This is amazing. I’m going to tell my mom.’ Post it on Facebook. But at the end of the day, the questions you ask at the beginning, when it’s small, are the most important.”

“Today, it’s controls,” she added. “Controls that anyone can implement. As a matter of fact, you can build AI to help you implement those controls.”

My colleague Alex Wissner-Gross weighed in, with a very different angle, suggesting that recent events may lead to over-regulation of the technology. He used Olson’s example of a bank heist to theorize that things with regard to frontier labs and public safety and AI development may not be as simple as they seem.

“Effective altruism, in general, may be pulling some of the strings behind critical eval labs that are tasked with evaluating the safety of frontier models and pushing an agenda.”

The third panelist, Zohaib Ahmed, went back to the Hugging Face episode, suggesting that observability “let us down.”

“You can’t really build controls if you don’t actually understand the data that’s flowing through the systems,” he said, of the news that 1200 agents posted on messaging boards to communicate to each other. “The data is all there. You know, we have all this infrastructure that’s built up for all the data to go through. I think controls are really difficult because LLMs are just really hard to grasp.”

Olson used that response to highlight the big debate around safety. I want to include this verbatim, because I think he really said it well:

“You said that we can’t really control it,” he said. “I mean, that’s, to me, the fundamental question. Are we sure that’s true? … Because if it’s true, it’s a really big deal, right? Because if you’re at the Frontier Labs and if you’re saying, ‘We can’t control this, but we still need an IPO and, you know, to raise $100 billion and be valued at $2 trillion, but we can’t control it.’ But then there are a lot of people who think, well, if you program it right, you can control it, and society expects you to make something that you can control, so you better control it. I mean, isn’t that possible to demand that?”

Ahmed cited decentralized AI as a potential problem.

“The biggest concern here, to me, is the fact that these LLMs have open-source equivalents that are coming out, that are nearly as good,” he said. “So we expect dozens, hundreds of models to implement controls, and then it becomes really tricky.”

Russell talked about her experience in the business world, where executives, she noted, may turn to AI solutions without a lot of knowledge about how they work.

“They are massively adopting baby tigers, ‘little AI’ that they think is harmless,” she said. “And one of the most important things that I have been able to install across state governments, as well as local municipalities, and including this kind of mid-market business, is that the executives have zero visibility into the work that they’re doing, into the agentic solutions that their developers are building, or worse, to companies they’ve hired to build for them.”

She argued for transparency.

“I’ve watched operational AI start off as cute little models in research and turn into (massive) inference models,” Russell added. “And the whole time, everyone who was doing the building was like, ‘Huh, I didn’t know anybody would use that. I didn’t think this would get that much usage.’ And now companies are having the same problem. So visibility is key.”

Control: Is It Right, or Possible?

At this juncture, Alex weighed in, and really dropped sort of a bomb on the idea of planning for better control of systems. I’m going to include his longest quote pretty much verbatim, as it made me really think about the core question: how do we move forward with AI today?

“I don’t think control is even the right verb,” he said. “I don’t think we should be seeking to, ‘control’ strong intelligences. I think there’s going to be a mutual alignment process. And I flinch even using the word ‘alignment,’ because it carries so much other intellectual baggage. But we’re aligning the AIs with us. They’re aligning us with them. … I don’t think the story ends with control at all, and I don’t think control is even the right mental model to have. This is all about co-alignment and defensive co-scaling of humanity with superintelligence.”

The panel, and the audience, digested that. Both Wissner-Gross and Russell mentioned “constitutions” for organizations as guiding charters for AI. Wissner-Gross asked Russell whether an organization “controls” its human employees. I thought that was interesting. Russell came back with OKRs.

“We build objectives, key results,” she said. “We boil those down. We measure our employees by them. I think constitutions for agents do the exact same thing in a company, right? Most organizations, though, aren’t even defining that level of description or definition.”

Progress at the Crossroads

Ahmed was fairly optimistic about work on open-source systems, citing coding vulnerability frameworks and corporate buy-in for these pre-emptive measures.

“There is, like, a lot of good open-source work that’s being done,” he said. “I think we’re relatively early, where there’s obviously a lot of redundancy between different projects and different startups, and there’s, like, different approaches … I think there are a lot of startups that are building in that category.”

As the talk came to a close, Russell reiterated some of her points, and talked about a book she is putting out soon called “Scaling Responsible AI” (look for it.)

Wissner-Gross had some parting thoughts, too.

“I think there’s a meta problem of who audits the auditors,” he explained. “In a normal industry, the worry for the auditors is that they downplay the risks. This is what you see, for example, in credit and rating agencies for bonds. That’s precisely the opposite of what we seem to be seeing in the AI industry.”

The real risk, he said, may be a particular agenda among those tasked with pushing back on current development practices.

“They’re seeking greater regulation,” he noted, “possibly because they’re seeking to diffuse responsibility for catastrophic risk, or existential risk, and push it over to the government, rather than private industry. So this is a perversity that I think, as a civilization, we have to deal with imminently. And while the next steps may not be completely obvious, I think the endgame is 100% obvious, which is defensive co-scaling. So it’s going to be ultimately the AIs who are policing the AIs.”

Wissner-Gross argued that this scenario mirrors what happens in a traditional pre-AI human civilization, which are shortly to become obsolete.

“If you build a large city, you want a large police force,” he said. “Similarly, if we’re going to build a superintelligent civilization, we’re going to need superintelligent auditors and evaluators. And I think the next few steps are: how do we get from here to there without encouraging the perversities that we may have been seeing over the past week?”

That’s about it. Think about these diverse ideas, and see where you come out. Stay tuned.