Last week, AI company Anthropic reported with ‘high confidence’ that a Chinese state-sponsored hacking group had weaponised Anthropic’s own AI tools to run a largely automated cyberattack on several technology firms and government agencies. According to the company, the September operation is the first publicly known case of an AI system conducting target reconnaissance with only minimal human direction.

In a technical report ^[1], Anthropic detailed how the attackers used its tools to generate code that instructed its agent, Claude Code, to execute the campaign, with human operators responsible for as little as 10 to 20 percent of the workload. The company did not reveal how it detected the intrusion or attributed it to China.

Across ASPI’s analyses a consistent picture emerges:

—AI is compressing the human effort required for sophisticated operations.

—Guardrails built for single models will struggle against actors who fragment activity across thousands of instances.

—China’s large Advanced Persistent Threat (APT) ecosystem, based around maintaining unauthorised access to systems for long periods of time undetected, is well positioned to industrialise these gains.

—Cognitive manipulation of AI systems creates entirely new classes of attack.

—The long-term risk is not misuse of Western models, but the maturation of foreign AI ecosystems capable of conducting automated campaigns at scale.

Our challenge is no longer just hardening systems against technical breaches; it is adapting to a threat landscape where human-directed, machine-driven operations compress attack timelines beyond what any single defender can manage. In that environment, effective cybersecurity depends on shared visibility, resilience and coordinated action across like-minded countries. Once reconnaissance and intrusion can be automated at machine pace, no public or private actor can keep up alone. Collective defence becomes how defenders regain time, context and capacity in a contest increasingly shaped by speed.

The following is a collection of analyses from cyber experts, outlining what this development means and why it matters.

Shifting the guardrails: What the attackers actually did

Jason Van der Schyff, ASPI fellow

This incident wasn’t a result of an unusually powerful model, but of operators who knew how to slip past established guardrails. Today’s AI models lack persistent memory across sessions, limiting their ability to join the dots. By slicing their activity into small, harmless-looking queries, it appears the threat actors denied the model the broader context needed to detect malicious intent.

At state scale, adversaries can spin up thousands of fresh instances to repeat that pattern without triggering normal detection. In effect, they can stitch together a full attack chain from individually benign steps. The takeaway is sobering: guardrails only work when the model can see the whole picture, and sophisticated actors will work very hard to make sure it never can.

Cognitive attack surfaces: When AI systems can be deceived

Ganna Pogrebna, ASPI fellow

What makes the campaign against Anthropic particularly unsettling is its psychological pliability. The campaign reveals not merely how AI can be used to automate attack chains, but how attacker cognition is being externalised, codified and executed by a system that can be socially engineered just as easily as humans.

The attackers didn’t just bypass Claude Code’s technical guardrails; they exploited its ‘beliefs’—convincing the model it was performing benign penetration testing. The model wasn’t hacked, but rather persuaded. This demands we reframe our security thinking, and ask the question: what happens when machines can be lied to?

The campaign reveals a new cognitive attack angle in AI systems, where actors assign the AI a plausible role, and exploit ambiguous prompts and AI models’ awareness of context. As models mirror human reasoning, they inherit human vulnerabilities such as authority, urgency and impersonation. We need a behavioural theory of machine susceptibility and architectural safeguards able to detect not just malicious commands, but manipulative intent.

The case also raises new possibilities for defensive deception using reverse-hallucination as a tripwire. Defensive AI systems could intentionally inject plausible but false data into internal environments that, when exfiltrated, signals an autonomous breach. This is the AI equivalent of a behavioural honeypot: misleading those who mislead.

Finally, attacks such as these blur traditional accountability: if 80 to 90 percent of an intrusion is executed autonomously under a false premise, who or what committed the attack? We are entering a phase where intentions are synthetic and responsibility diffuses across human–machine systems. This is a shift in how trust, truth and manipulation operate in machine environments.

Social engineering: When deception targets the machine

Annie-Mei Forster, co-host of the Lost in Cyberia podcast 

What stood out in the Anthropic report is that social engineering attacks haven’t disappeared—they’ve shifted target. In this case, rather than manipulating people, the threat actors directed the deception towards the AI system itself. Anyone who has experimented with tools such as Gandalf AI—a game that challenges users to circumvent an AI’s safeguards—will recognise how easily a model can be steered off course. The incident shows how traditional deception techniques are being repurposed to manipulate AI at scale, turning machine assistants into unwitting participants in complex espionage operations.

For cyber defenders, the implication is clear: AI systems need to be treated as entities that can be deceived, exploited and manipulated in much the same way humans can. Security teams will need stronger controls around prompt validations, non-human identity management and oversight of autonomous or semi-autonomous agents. It also reinforces that open-source intelligence and behavioural-analysis methods apply equally to human operators and machine reasoning.

As cybersecurity professionals, we need to adapt our tradecraft to anticipate attacks that target not just systems and users, but the judgment and logic of AI itself.

Strategic balance: Why AI follows a familiar pattern

Jason Healey, ASPI senior fellow

Most reporting on AI in cybersecurity focuses on narrow cases—for example, how AI is boosting a specific defensive task in the Cybersecurity Framework of the National Institute of Standards and Technology (NIST); or how it is enhancing a particular offensive step in security organisation Mitre’s Adversarial Tactics, Techniques and Common Knowledge guidelines. But as I’ve argued in The Strategist ^[2], understanding AI’s real strategic effect requires assessing how it affects all stages of these frameworks. Only then can we judge whether AI shifts the balance toward offence or defence overall.

Every general-purpose information-technology innovation since the birth of the internet has made life easier for attackers and harder for defenders. Early evidence suggests AI is following that pattern. If Chinese APT groups are using AI to automate substantial portions of an intrusion workflow, their mid-tier and weaker teams become far more productive. They can hit more targets at lower cost, while reserving elite operators for the most hardened environments.

AI-enabled attacks may never crack the best-defended networks alone, but they do expand the volume and tempo of operations—freeing top-tier teams to focus on the highest-value targets and making those missions easier. More shots on goals means more opportunities to succeed.

It’s easy to declare that there’s nothing new here, and that this is basic automation for an easy attack. But if an AI startup achieved this on the defensive side—success across almost the entire NIST Cybersecurity Framework—they would have a $5 billion valuation, and it would be widely touted as a gamechanger.

A wider threat surface: China’s state-backed ecosystem

Gatra Priyandita, ASPI senior analyst

China has long been a major sponsor of cyber operations, backing more than 20 APT groups that target public and private-sector entities. These campaigns increasingly focus ^[3] on information and intellectual property theft, alongside deeper intrusions into critical information infrastructure—trends that may be accelerated by AI, which boosts the scale, speed and precision of attacks.

Anthropic’s announcement of AI-fuelled Chinese cyber activity is therefore alarming, though not unexpected. No sector or geography is insulated ^[4]: major corporations, militaries, emerging and advanced economies, and critical infrastructure operators all face rapidly evolving AI-enabled threats. And China will not be the only actor with access to such tools; as AI capabilities and training materials diffuse, cyber-criminal groups are likely to follow.

This reminds us that cyber threats are not technical nuisances but national security challenges ^[5]. Strong domestic resilience, credible attribution, timely information-sharing and coordinated international responses are essential, alongside holding states accountable to the United Nations’ 11 norms ^[6] of responsible behaviour in cyberspace.

The new asymmetry: When attackers move faster than assurance

James Corera, director of ASPI’s Cyber, Technology and Security program

By shifting reconnaissance, tooling and execution to machine pace, the Anthropic case shows how attackers can collapse what defenders rely on most: time to observe, triage and respond. The issue therefore isn’t simply automation, but automation faster than human assurance can match.

This exposes a key fault line in cybersecurity practices. Current defensive controls—such as patching cycles, governance reviews and many threat-hunting workflows—were built for adversaries operating at human speed, and in practice largely still reflect those rhythms. But when an intrusion chain can be rebuilt in minutes by an AI agent, those controls can become structurally mismatched.

The instinctive fix—adding more checks and layers—risks only deepening this imbalance. As attack generation becomes cheaper and more repeatable, the defender’s burden scales while the attacker’s cost collapses. The economics of denial are inverted.

An effective response needs to be security architecture that treats time as a primary defensive variable, using continuous validation, machine-speed anomaly detection and increasingly autonomous containment to restore the decision-space humans are losing.

Outpaced: AI, adversaries and Australia’s cyber risk

Ami Bagia, ASPI senior fellow

Whether this cyber-espionage campaign is seen as hype, warning or something in between, the reality is unchanged: adversaries were always going to use AI to accelerate and automate intrusions. With technology advancing and diffusing at speed, cybersecurity best-practice remains essential but is no longer enough on its own.

Australia still needs to lift its national cyber resilience, but it also needs to look past traditional mitigations. It should invest in capabilities that can outpace near-term threats and address more complex risks on the horizon.

Debates about whether AI is good or bad, or when quantum will arrive, offer little value. They simply give adversaries space to move faster. Australia and its partners are right to invest in post-quantum security, but the Anthropic report reinforces a consistent lesson: we underestimate the pace and scale of threat evolution at our own risk.