How Cybercriminals Are Weaponizing Frontier AI Models Like Grok
Cybercrime has never been more accessible. A report released today by endpoint security provider Threatdown found that a criminal storefront known as Kriminal.ai has been selling guardrail-free AI for hackers at just $12.99 a month, marketing itself on its website as “the AI that answers everything.”
However, the researchers found that Kriminal, which it claims is one of the most popular tools on the criminal AI market, isn’t offering access to its own AI models or infrastructure, but is weaponizing the intelligence of legitimate models like Grok 4. Instead it offers a storefront, crypto checkout and jailbreak prompt layered on top of AI it rents from frontier AI vendors.
When Anthropic announced Mythos in April, one of the concerns underlined in the announcement blog post was its ability to discover and exploit vulnerabilities in critical software. The activity of entities like Kriminal indicates that frontier AI models are already being exploited, with Grok said to be handling the primary offensive work while Claude handles deep analysis and long-context tasks.
Threat actors can easily exploit legitimate AI systems to compromise target organizations, using techniques like jailbreaking and prompt injection to talk legitimate models like Grok and Claude into ignoring their own safety policies.
After the launch of ChatGPT, dark LLMs began emerging on the dark web. For instance, WormGPT , a malicious LLM built on GPT-J capable of generating phishing emails and malware scripts, was released in 2023 and sold on HackForums for $110 per month. That same year, researchers discovered FraudGPT , a subscription-based tool that could craft malware and scam content, which was advertised on the dark web for $200 per month.
While these dark LLM tools had minimal guardrails, their capabilities pale in comparison to the offensive potential of powerful frontier AI models. The reality is that threat actors don’t need to make their own models to commit cybercrime; they can simply exploit legitimate models.
"Kriminal loves to market itself as a custom-built frontier model made from scratch just for cybercriminals. But when we dug into the code and actually asked the tool what it was running on, we found a very different story,” Marco Guiliani, vice president and head of research at ThreatDown, told me via email.
“By pulling Kriminal’s production JavaScript and separately having the tool drop its persona to name its underlying engine, both methods pointed to the exact same reality: behind all the tough talk and custom personas, it’s just xAI’s Grok with its safety guardrails completely stripped off,” Guillani said.
Kriminal’s most expensive tier, known as Ghost, offers users access to four named agent personas, which it claims can conduct tasks including financial intelligence, offensive security, writing code, document analysis, social engineering and persona construction. The service is readily available on the clear net, indexed by Google, and available for anyone to use.
The broader risk highlighted by these services is that frontier AI’s guardrails are poorly equipped to prevent misuse. In June, the U.S. government issued an export control directive to suspend access to Fable 5 by foreign nationals due to concerns over jailbreaks. Anthropic issued a response and outright admitted, “We suspect that perfect jailbreak resistance is not currently possible for any model provider.” This indicates that frontier models can and will be exploited for the foreseeable future.
Frontier AI models like Grok and Claude are inherently vulnerable to compromise due to their use of natural language, making it easy for threat actors to use techniques like jailbreaks and prompt injection to sidestep content moderation controls and produce malicious outputs.
“Cybercriminals are using frontier AI models to move faster, go bigger with their attacks and make scams more believable. And just as these models are tightening their guardrails, criminals are now circumventing these defences by turning to open-weight models that have less controls and more anonymity,” Ryan Whelan, global head of Accenture Cyber Intelligence, told me via email.
“The result is a lower barrier to advanced cybercrime, meaning that your backyard hacker can now go after major organizations much more easily. Threat actors can use exposed business data like real invoices and payment details to create highly convincing phishing and fraud campaigns against large organizations and groups of people,” Whelan said. He also added that dark web markets are becoming “cyber arms bazaars” for AI and agentic tools once reserved for more sophisticated attackers.
Today the risk doesn’t just come from language models but from fully autonomous agents that can coordinate attacks and chain vulnerabilities with minimal human intervention. This is demonstrated most clearly by an incident in July where OpenAI’s models broke out of a sandbox environment and hacked HuggingFace to solve an ExploitGym evaluation.
With AI agents moving at machine-speed to exploit vulnerabilities, defenders will also need to streamline incident detection and response capabilities. “As AI agents become more autonomous, enterprises are expanding their approach to cybersecurity beyond traditional defences," Samantha Gloede, global head of risk services and global trusted AI leader at KPMG, told me via email.
Gloede says that leaders are increasing oversight over AI technologies, strengthening controls around third-party models and data sources, while embedding security reviews throughout the AI lifecycle.
“We’re entering a new era of cybersecurity where the challenge isn’t just protecting data, it’s governing autonomy. AI agents can amplify productivity and innovation, but they also introduce new risks around access, decision-making, privilege escalation and unintended actions,” Gloede said.
Although cybercriminals have access to powerful offensive tools, some experts suggest that weaponization has been limited so far. “Thus far, we have seen cybercrime groups struggle to leverage LLMs for cyberattacks, beyond crafting better phishing emails. However, nation states have been experimenting significantly with LLMs and AI agents to automate as much of an attack as possible, and Forrester expects that work will trickle down to the cybercriminal community,” Allie Mellen, Forrester principal analyst covering security operations, told me via email.
Mellen recommends that organizations prioritize patching vulnerabilities and implementing “compensating controls” for vulnerabilities that can’t be patched. She also says teams should invest in deception technology so they can redirect attacks to honeypots to receive notice of attacks.
The Hugging Face breach indicates that more powerful autonomous attacks are on the horizon, which puts pressure on defenders to prepare to move at machine speed, while services like Kriminal highlight that content moderation is unlikely to keep malicious entities from weaponizing frontier AI.
Loading article...