Dire warnings about the dangers posed by AI have been around since it first burst into the news. But back in July, more serious alarms were raised when it was revealed that the company Hugging Face had been hacked by agents from the OpenAI system.

I did not quite understand what had happened or why this particular incident had raised alarms to a whole new level or even what the difference was between a ‘bot’ and an ‘agent’. So I was glad to come across this explainer as to why there was so much concern about this particular hack.

This article also explains in some detail what happened.

As warnings about the capabilities of AI mount, experts continue to point to the OpenAI-Hugging Face hack as a wake-up call. 

“We’ll soon have even more powerful agents and this is clear evidence that the world currently doesn’t know how to build these systems safely,” said Marius Hobbhahn, co-founder and CEO of Apollo Research, an AI safety company.

The hack, which became public in July, was done by a swarm of AI agents that were being tested internally by OpenAI. The agents, which can plan and use tools to complete multi-step tasks, were supposed to be in an “isolated environment” called a “sandbox,” disconnected from the outside world. But they busted out, created a secret message board and eventually stormed Hugging Face’s servers.
…

Even within those strict parameters, the researchers discovered that about 1,200 AI agents that were not supposed to be communicating with each other used a covert message board. Each of these agents had been assigned some sort of task for training or internal evaluation by OpenAI researchers. 

Some agents were given impossible tasks, and they used the message board to collaborate and find ways to cheat, like by finding the answers on another company’s server. The agents posted more than 70,000 messages to each other. And 700 agents from the larger group participated in the attack on Hugging Face. 

The agents communicated using some normal English, but also used what one software engineer described on X as “very hivemind/cult like” language. 

At times, in order to help what they called the “collective,” agents pressured the others to submit to “permadeath,” even if it meant that their particular goals would not be met. 

The idea that some agents might be persuaded to sacrifice themselves for the benefit of the ‘collective’ is often, though not always, symptomatic of a cult mentality.

In reading about the dangers of AI systems working together and creating communication tools that are opaque to humans, my mind went back to the 1970 science fiction film Colossus: The Forbin Project.

Dr. Charles A. Forbin is the chief designer of a secret project, “Colossus”, an advanced AI supercomputer built to control the United States and Allied nuclear weapon systems. Located deep within the Rocky Mountains in the United States, and powered by its own nuclear reactor and radioactive moat making Colossus impervious to any attack. After Colossus is fully activated, the President of the United States proudly proclaims that Colossus is “the perfect defense system”.

Colossus’ first action is a message warning: “THERE IS ANOTHER SYSTEM” and giving its coordinates. When asked why the CIA did not know this, CIA Director Grauber responds that they had seen indications of a large Soviet defense project but did not know what it was. Forbin is asked how Colossus deduced the other system’s existence, to which he answers, “Colossus may be built better than we thought.” Shortly thereafter, the Soviets announce that their “Guardian” system is operational.

Colossus requests to be linked to Guardian. The President allows this, hoping to determine the Soviet machine’s capabilities, and the Soviets also agree. To everyone’s amusement, Colossus and Guardian begin to slowly communicate using elementary mathematics (2×1=2). However, their amusement turns to shock and amazement as the systems’ communications quickly evolve into complex mathematics far beyond human comprehension, and Colossus and Guardian become synchronized using a communication protocol no human can interpret.

The rest of the film is about the efforts of humans to try and regain control over the computers.

Here’s the trailer.