Cybersecurity Risks Posed by Frontier AI Models

Almost everything we do today depends on computers in one way or another. We pay for our purchases, plan our day, contact our loved ones, and even drive our cars with the help of computers. And nearly all these computers are connected to one another. This makes us all vulnerable to cyberattacks.

Highly effective cyberweapons, malware, and botnets — such as Stuxnet, Mirai, and EMOTET — have always been difficult to create. For example, developing the Pegasus cyberweapon cost hundreds of millions of dollars. Discovering so-called zero-day exploits (vulnerabilities that have not yet been discovered) requires considerable skill and time; only highly specialized hackers can do this. However, once artificial intelligence becomes sufficiently advanced, this may no longer be the case. Instead of hiring a team of security experts or highly skilled hackers to find zero-day exploits, anyone could simply use a much cheaper AI system.

AI Models Can Autonomously Find and Exploit Vulnerabilities

The latest AI systems can already analyze and write software. They can identify software vulnerabilities and be used to exploit them.

GPT-4 can already hack websites autonomously, performing complex tasks such as blind database schema extraction and SQL injection without human feedback. This capability was discovered 18 months after GPT-4 completed training. GPT-4 already outperforms 88% of human hackers in a Capture the Flag (CTF) competition.

It can also autonomously exploit 87% of the tested vulnerabilities, representing a significant leap forward compared with GPT-3.5 and open-source models, all of which scored 0%.

Teams composed of multiple large language models (LLMs) perform even better, demonstrating the ability to exploit real-world zero-day vulnerabilities.

As AI capabilities increase, so will the number of vulnerabilities these systems can detect and the exploits they can develop.

For now, they are not as capable in this area as the best human experts, so the immediate danger remains limited. However, these capabilities are advancing rapidly and may improve in sudden leaps.

It is worth noting that AI also enables entirely new types of attacks. For example, AI can be used to infer a password from the sound of someone typing during an online call, or to use Wi-Fi signals to detect people through walls. AI can also be used to create self-modifying malware, making it much harder to detect.

We will probably reach a point where artificial intelligence is better at hacking than the best human hackers. This could go wrong in several ways:

Infrastructure: Cyberweapons could be used to gain access to or disable critical infrastructure, such as oil pipelines or power grids.

Finance: Cyberweapons could be used to steal money from banks or manipulate the stock market.

Military: Equipment such as weapons and sensors increasingly relies on wireless connectivity and complex software.

Large-Scale Cyberattacks

Such powerful AI could potentially be used to create a virus that exploits a large number of zero-day vulnerabilities. A sufficiently capable AI system could analyze and identify vulnerabilities in the source code of every operating system and other software. Such a virus could infect any computer, regardless of its operating system, through multiple channels, including Wi-Fi, Bluetooth, and Ethernet (UTP). This could give the attacker complete control over these devices, allowing them to steal data, use their hardware for their own computing tasks, encrypt their contents for ransom, or disable them entirely.

A virus like this could be created by criminals as a tool for stealing money or developed by a nation-state or terrorist organization as an extremely destructive cyberweapon. However, as AI becomes more agentic — capable of acting autonomously — such a virus could also be created and deployed autonomously by an AI system misaligned with human values.

If the goal of a cyberattack were to disable devices and infrastructure, the damage could be massive. Our society increasingly depends on computers and the internet. Payments, transportation, communication, planning, supply chains, power grids — if our devices stopped functioning properly, many sectors of society would also cease to operate.

More than 93% of cybersecurity experts believe that “a catastrophic cyber event with widespread impact is likely within the next two years.”

Mitigating AI-Related Cybersecurity Risks

The scenario described above could occur only if:

1. The ability to discover zero-day exploits emerges. Current models can already discover some vulnerabilities, and this capability will most likely improve with future models.

2. The model falls into the hands of malicious actors. This could happen if the model weights are leaked, the model is released as open source, or it is developed by a malicious actor.

3. Security vulnerabilities are not patched before such a cyberweapon is deployed. Unfortunately, defenders may be at a disadvantage if the model is widely distributed, for two reasons:

• Developing fixes, releasing patches, and deploying them takes much longer than launching an attack. The window of vulnerability is longer than the time required to create the attack.

• Attackers need to find only one vulnerability, whereas defenders must find and address all of them.

There are several measures we can implement to combat these risks:

Prohibiting the training of models capable of discovering zero-day exploits.

This is the most effective way to prevent this scenario. It is the safest approach and the one we propose.

Allowing models to be released or published as open source only after extensive testing.

If models possess dangerous capabilities, they should not be released.

Imposing strict cybersecurity regulations to prevent model-weight leaks.

If dangerous models are allowed to exist, ensure that they do not fall into the wrong hands.

Requiring AI companies to use AI to remediate vulnerabilities.

If a model capable of discovering previously unknown security vulnerabilities is trained, use it to contact the maintainers of the affected software so that the vulnerabilities can be fixed. Allow sufficient time for patching before releasing the model. Ensure that its weights are not leaked and protect the model as if it were the launch code for a nuclear attack.

If done correctly, AI could dramatically improve cybersecurity across the world.