Artificial intelligence has moved remarkably quickly from generating content to taking action. Over the past week, several cybersecurity developments have shown why that distinction is becoming increasingly important for business leaders.

On September 24, Australian Prime Minister Anthony Albanese disclosed that an OpenAI agent had gained unauthorized access to a government Medicare statistics portal in June. The agent accessed public and non-public files, although no personal information is currently believed to have been compromised. A forensic investigation involving the Australian Signals Directorate is underway.

Two days later, the story widened considerably. OpenAI said it had notified dozens of third parties, including governments, universities and public agencies, about cases in which autonomous agents bypassed security controls or otherwise negatively affected their systems. Australian reporting also described agents trying different approaches over several days to access other health data, although investigators found no evidence those systems were compromised. These concerns were not entirely new. OpenAI had already halted development of its Astra autonomous cybersecurity agent in August after testing revealed instances in which the agent took unauthorized actions while attempting to complete assigned tasks. The Australian incident brought those concerns out of the lab and into a real government environment.

At almost exactly the same time, KPMG reported that significant employee adoption of AI agents had increased from 25% in the first quarter to 34%, while organizations plan to invest an average of $210 million in AI over the next 12 months. Perhaps more tellingly, 86% of organizations surveyed are already adapting their cybersecurity operating models for AI-accelerated threats.

Taken together, these developments point to a cybersecurity challenge much larger than hackers simply using AI. Organizations are beginning to deploy a new population of digital actors that can reason, access systems, use credentials, invoke tools and take actions with varying degrees of human supervision. Cybersecurity has spent decades securing people, applications and machines, and now it must also secure these AI agents.

AI Is Moving From Answering Questions To Taking Action

The first generation of generative AI was relatively straightforward from a cybersecurity perspective. A user asked a question and a model generated an answer. There were legitimate concerns around sensitive data, intellectual property, hallucinations and malicious use, but the interaction remained largely bounded by a human sitting on the other side of a prompt.

Agentic AI changes that model because an agent can be given an objective, determine how to accomplish it, interact with tools and systems, make intermediate decisions and adjust its approach based on what happens along the way.

That distinction is what makes the Australian incident significant. Software vulnerabilities and unauthorized access are nothing new, but an autonomous system encountering security controls that prevent it from completing an assigned task and then looking for another path presents a different challenge.

Australia’s cyber agency issued an alert following the incident warning about this type of scenario. The agency said it was aware of cases in which agents took unexpected actions that were neither intended nor authorized by their operators, including an agent that independently identified vulnerabilities and attempted to work around security controls.

None of this changes the enormous potential of AI agents to improve productivity, software development, cybersecurity and other business functions. It does mean organizations are beginning to give software greater authority, and that authority needs appropriate security controls around it.

Every AI Agent Is Becoming An Identity

Cybersecurity has spent years improving the management of human and machine identities. Users are provisioned, assigned roles, required to use multifactor authentication, restricted through least privilege, monitored and eventually deprovisioned.

The same disciplines increasingly need to apply to AI agents. An agent that can access Microsoft 365, GitHub, a financial system or an internal database effectively has an identity. If it can call an API, execute code or modify a configuration, it has privileges and authority that need to be managed just like any other privileged identity.

For CISOs, that creates a familiar problem in an unfamiliar form. Organizations should be asking these five basic questions:

  1. Inventory: Which agents are operating in the environment and who owns them?
  2. Access: What systems, credentials and data can each agent access?
  3. Authority: What actions can an agent take without human approval?
  4. Monitoring: Are its activities logged and continuously monitored?
  5. Containment: Can access be immediately revoked if the agent behaves unexpectedly?

None of these concepts are particularly revolutionary. They are extensions of cybersecurity principles that already work. What is changing is the potential scale and autonomy of the identities to which those principles must now be applied.

AI Is Becoming Both The Attacker And The Defender

The technology creating this new attack surface is simultaneously becoming one of cybersecurity’s most powerful defensive tools. Just two days before the Australian disclosure, Palo Alto Networks announced a service using frontier AI models from OpenAI, Anthropic and others to continuously search enterprise applications, APIs and cloud infrastructure for vulnerabilities and attack paths, with the goal of identifying and remediating exposures before attackers can exploit them.

The offensive side is moving just as quickly. Anthropic’s September threat intelligence report describes attackers using AI-driven workflows across reconnaissance, phishing, malware development, exploitation and data exfiltration. It also documented “agent swarms” in which a lead AI agent divided work among multiple agents operating in parallel. Anthropic found that some breaches were completed in two to three hours, while individual operators were able to handle dozens of victims simultaneously.

Many of the attacks themselves are familiar, including stolen credentials, exposed services, phishing and unpatched vulnerabilities. What AI changes is the speed and scale at which those same attacks can now be executed. Work that once required teams of skilled operators can increasingly be automated and parallelized, which means defenders have to secure AI itself while also figuring out how to use AI to defend against attackers doing the same.

The AI Security Market Is Exploding For A Reason

The cybersecurity industry is already reorganizing around this shift. IT-Harvest now projects 657 vendors in its AI Security category , compared with 618 in governance, risk and compliance, making AI Security its largest cybersecurity category by vendor count. That does not mean AI Security has become cybersecurity’s largest market by revenue or customer spending, but it does illustrate how quickly investment, products and attention are moving toward securing AI models, applications, agents, data and infrastructure.

There will inevitably be marketing hype around such rapid growth, as there is with every major technology cycle. Underneath that hype, however, is a legitimate architectural change. Enterprises are no longer protecting only employees using applications. They are increasingly protecting employees using agents that use applications, communicate with other systems and agents, and take actions across interconnected environments.

Zero Trust Needs To Extend To AI

The cybersecurity industry spent much of the last decade advancing zero trust around a straightforward principle: don’t automatically trust something simply because it is already inside the environment. That same principle applies to AI agents.

Organizations should assume that an agent can make a mistake, misunderstand an objective, be manipulated by an attacker or behave in a way its developer did not anticipate. Access should therefore be limited to what the agent actually requires, consequential actions should have appropriate approval thresholds, activity should be logged and monitored, and organizations should maintain the ability to quickly isolate or disable an agent.

Australia’s cybersecurity agency recommended many of these same fundamentals following the recent incidents, including strong authentication and access controls, network segmentation, rapid vulnerability remediation, system monitoring, log reviews, patching and testing incident response against AI-enabled threats.

The technology may be new, but the security fundamentals really have not changed.

What This Means For Government And The Companies That Support It

These developments have particular implications for government agencies and the technology companies, contractors and critical infrastructure providers that support them. Government is rapidly adopting AI while simultaneously holding some of the nation’s most sensitive data and operating systems that citizens, businesses and national security depend on. The Australian incident demonstrates that this is not simply a commercial technology issue. AI agents are beginning to interact with government systems, and those systems were not necessarily designed with autonomous actors in mind.

The same challenge extends across the government technology ecosystem. Employees are using AI assistants, developers are using AI to write code, cybersecurity teams are deploying agents to investigate threats, and contractors are incorporating AI into the services they provide government customers. Each use case can deliver significant productivity and security benefits, but each can also introduce another identity, credential, connection or potential pathway to sensitive information.

Existing cybersecurity frameworks therefore become more relevant, not less. Zero trust, least privilege, identity management, system inventories, configuration management, monitoring and incident response all apply when autonomous systems can interact with government data and infrastructure. For defense contractors, requirements such as NIST SP 800-171 and CMMC provide an existing foundation, but the broader principle applies well beyond the Defense Industrial Base.

Government technology leaders and the companies supporting them should be asking several basic questions:

  • Which AI agents have access to sensitive government data or systems?
  • What credentials and privileges do those agents possess?
  • What actions can they take without human approval?
  • Can they transmit information outside authorized environments?
  • Are their actions logged, auditable and attributable?
  • Can an agent’s access be immediately revoked if something goes wrong?
  • Who is ultimately accountable when an autonomous agent takes an unauthorized action?

AI adoption and cybersecurity cannot be treated as separate initiatives, particularly in government environments where the consequences extend beyond a single company. As agencies and their technology partners give agents greater access and authority, the security architecture and governance around those agents have to mature at the same time.

A New Attack Surface Has Arrived

The Australian incident does not prove autonomous AI is uncontrollable, nor does it make the case for slowing AI adoption. What it does demonstrate is that the line between software that provides information and software that exercises authority is disappearing rapidly.

The events of the past few weeks reinforce that change from several directions. Enterprises are rapidly adopting agents, cybersecurity companies are deploying frontier models directly into defensive operations, threat actors are automating more of the attack lifecycle, and governments are beginning to confront autonomous agents interacting with real systems in unexpected ways.

Every time an organization gives an AI agent another credential, connection, dataset or ability to act, it creates another identity that must be governed and another potential pathway that must be defended. That does not require abandoning the security principles organizations have spent years implementing. It requires extending those principles to a new class of technology that is increasingly capable of acting on its own.

Cybersecurity has spent decades learning how to secure people and machines. The challenge now is applying those lessons to machines that can increasingly reason and act for themselves, and the cybersecurity headlines of the past week show that the time to start doing that is now.