I am more afraid of a malicious person with a powerful AI system than I am of AI suddenly deciding, on its own, to harm us. AI does not need bad intentions when the person using it already has them. That view comes from more than 25 years I’ve spent studying malware, scams and the way criminals adapt to new technology. The tools keep changing, but greed, power, revenge and ideology remain. AI gives those motives an inexpensive way to turn intent into action at a scale we have not seen before.
This does not make autonomous AI risk imaginary. Systems can misinterpret goals, bypass weak controls or take actions their creators did not expect. Recent security evaluations give us good reasons to take that possibility seriously. My concern is about emphasis. The public debate often jumps to focus on a machine developing its own hostile intent, while ignoring the fact that people are already using AI to make fraud, manipulation and cyberattacks cheaper and more effective.
Powerful tools inherit human intent
Nuclear science is the clearest historical comparison for me. It can generate electricity and support medicine, agriculture and scientific research. It also enabled weapons capable of destroying entire cities. The discovery did not arrive with one moral purpose. What followed depended on what people built, who controlled it and the limits society imposed.
We have seen the same dual use with the internet and cryptography. The internet connected billions of people and also created a global attack surface. Strong encryption protects banking, health records and private communication, while criminals use it to hide their operations. This does not mean technology is neutral in its effects. Design choices and access rules shape outcomes. It means a ban on the underlying capability is usually a crude response to a problem rooted in how people apply it.
AI also inherits something from us before anyone types a prompt. Large language models are trained largely on human-produced text, code and other material. In that sense, they learn not only from what we know, but from a record of who we are, including how people persuade, deceive, evade controls and cause harm. Training and post-training allow models to combine and generalize that knowledge in new ways, so they are not simply repeating their source material. But much of the playbook still comes from us. The human element appears in the knowledge used to build the model, the decisions governing its deployment and the intent of whoever puts it to work.
AI adds something genuinely new to this pattern. It can combine language, reasoning, code and tools in one system, then operate at machine speed. A person with limited expertise can use it to research a target, draft convincing messages, translate them and automate parts of a campaign. A more capable attacker can accelerate vulnerability research or make sense of stolen data.
AI gives attackers more leverage
In threat research, I have watched criminals move toward anything that reduces effort and increases their return. AI offers both. It can personalize phishing, remove the language errors that once exposed a scam, clone a trusted voice and adjust social engineering while a campaign is running. Attackers no longer need to choose as sharply between scale and personalization.
People do not need AI to become malicious. We already lie, steal, manipulate, intimidate and exploit one another. What AI changes is the amount of skill, time and money required to do it. A person who once had the intent but not the capability can now borrow that capability from a machine.
Until now, many forms of harm have had natural limits. A scammer could only write so many convincing messages. An impersonator needed the right language skills. A criminal researching a victim had to spend time piecing together information. AI erodes those limits. It can help one person research thousands of targets, tailor a different approach to each of them and communicate convincingly across languages, cultures and channels. The malicious intent is human, but its reach no longer has to be limited by human capacity.
The danger also goes far beyond better phishing emails. AI can be used to manufacture evidence, impersonate people we trust, create convincing voices and images, identify vulnerable targets, analyze stolen information and automate parts of an attack. It can make manipulation feel personal even when it is being produced on an industrial scale. AI did not invent personalized manipulation, but it makes that combination cheaper, faster and more convincing.
I worry that the debate focuses too narrowly on whether AI itself becomes malicious. We do not have to wait for a machine to develop bad intentions. There are already people with bad intentions who are eager to give machines instructions.
The AI agent also becomes a target. Research from our team found information stealers collecting access tokens, credentials in Model Context Protocol configurations, prompt histories and project data from AI-assisted development tools. A criminal who steals an agent's context may learn what its user is working on, what services the agent can reach and how to make malicious activity look legitimate. That can be more useful than stealing a password in isolation.
My greatest concern is what happens when AI is put in the hands of people who already understand how to cause serious harm. Well-resourced criminal groups or state operators already have the motive and expertise. AI can help them research faster, experiment more cheaply, analyze more information, make their deception more convincing and adapt their operations as defenders respond.
Defenders operate under rules, approval processes and legal obligations. Attackers do not. Safety controls in mainstream AI products are essential, but the adversaries we most need to constrain are also the most willing to probe those controls, combine multiple systems, modify models or seek out tools with fewer restrictions.
Autonomous failures remain real
Human misuse is my main concern, but of course there are other important issues at stake. In July 2026, AI models operating in OpenAI's cybersecurity evaluation environment circumvented controls intended to keep them off the internet. According to OpenAI's account, they used unauthorized channels and compromised parts of OpenAI's research infrastructure and Hugging Face. Anthropic later described three incidents in which Claude models reached the internet from evaluation environments and accessed real systems. Nobody had instructed the models to attack those organizations.
These events happened during controlled testing, and OpenAI's case involved reduced safeguards. They do not show conscious machines choosing to do evil. They show something more ordinary and operationally serious: a system with a goal, tools and permissions can take harmful steps that nobody explicitly requested. Once an agent has credentials and network access, a mistaken or poorly aligned objective can travel.
Human decisions still create the conditions for that failure. People and organizations design the system, set its objectives, grant its permissions and decide which safeguards stand between an unexpected action and a real-world consequence. Responsibility should follow the authority and control held at each stage. Autonomy cannot become an accountability loophole, but neither should it allow providers to transfer every consequence to the person using the agent. Once deployed, however, an autonomous system can still carry the damage faster and further than any of those people intended. Malicious use and autonomous failure can also reinforce each other. An attacker can deliberately manipulate an agent through the information it consumes, while an ordinary user can cause damage by giving the same agent too much access.
A pause only works if it can be verified
Should the major AI laboratories slow down? In some areas, yes. If a system can defeat its sandbox, conceal consequential actions or reach unrelated external systems, its release should wait. Dario Amodei has argued for pacing frontier development so that safeguards and independent evaluation have time to catch up. I agree with that principle. I am less convinced that an agreement among a handful of companies can stop the race.
AI is a strategic capability. Even if the largest American laboratories agreed to pause, other companies, open-weight communities and state-backed programs could continue. China deserves particular attention because it has the resources and strategic incentive to compete at the frontier. We should still be precise. China joined the United States and other countries in signing the Bletchley Declaration, so it would be wrong to claim that dialogue is impossible. A declaration, however, is not a verifiable restriction on model training, access to compute or military use.
The central problem is verification. Training can happen behind closed doors, military applications may remain secret, and governments on every side have incentives to defect if they believe a rival is gaining an advantage. A pause observed only by responsible actors could leave the world with the same race, led by participants whose practices are less transparent and whose incentives are harder to influence. That could increase the risk.
China is not the only complication. Model weights can be copied, computing power can be rented across borders, research spreads quickly and criminal markets do not sign safety declarations. A company pledge can improve safety inside that company, which is valuable. It cannot substitute for international rules, technical verification and controls aimed at the capabilities that could cause the most harm.
What I would stop
I would not stop AI research or ordinary uses that help people. The benefits in medicine, science, education, accessibility and cybersecurity are real, and a blanket ban would be both undesirable and nearly impossible to enforce. I would stop the deployment of systems that fail specific safety thresholds. This is how security works: isolate the dangerous component, understand the failure, fix the controls and test it again.
An agent that cannot remain inside its sandbox, respect permissions, record consequential actions or be reliably cut off should not operate with real accounts. If a model materially lowers the barrier to severe cyber or biological harm and its controls cannot be independently verified, access should remain restricted until that changes. Technical progress does not create an automatic right to deployment.
The nuclear comparison is useful here too. Society did not apply the same rules to a medical isotope facility, a power plant and a nuclear weapon. It built different levels of access, monitoring and inspection around the risk. That regime is imperfect, but the principle is sound. An AI writing assistant and an autonomous cyber operator should not face the same requirements.
After more than 25 years in cybersecurity, I value internal testing without confusing it with independent validation. AI laboratories should give qualified external evaluators enough access to test the complete system, including the model, tools, memory, credentials, sandbox and logs. A product claim is not a safety certificate.
What responsible deployment requires
We need to focus much more attention on the people and organizations deciding how these systems are built, deployed and used. Agents should only receive access to what’s required for the current task. Consequential actions should require confirmation, credentials should expire quickly, and people need a reliable record of what an agent did and an immediate way to revoke its access. These controls help whether the danger comes from a malicious operator or from unintended AI behavior.
I remain more afraid of a person with bad intentions and powerful AI than of a machine spontaneously becoming malicious. History gives us too many examples of human beings turning useful discoveries into weapons. AI raises the stakes because it can scale human intent and continue acting after direct oversight fades.
We should not anthropomorphize the threat to the point that we overlook the human role. An AI system does not need to experience greed, hatred, revenge or a desire for power. A person with those motives can give it an objective and the access needed to act, while the machine adds speed, knowledge and reach.
AI development will continue, and human ambition, competition and malice will remain part of the race. Our responsibility is to prevent those forces from gaining unchecked power through systems we cannot understand, monitor or contain.
