I’m one of the least qualified people in the world to be talking about this. I might be the fourth or sixth best qualified on the network and nobody better has expressed interest in it, so yer gettin’ what yer gettin’. Swarms and fleets and as yet undescribed deployment of agentic LLMs! I’ll say a few things about them and try not to get it completely wrong, but the way I’ll achieve this without doing a ton of research is by hardly saying anything at all. Here goes.
They made LLMs by raising the limits on smaller language models, right? It’s all weighted probabilities of word or word portions emerging from a pile of math. The expected result was that they’d be better at acting like a human for the purposes of verbal interaction, but they exceeded expectations dramatically. Even experts have serious disagreement about the extent to which they have become thinking entities. Man, choosing that phrase “thinking entities” was a challenge in itself. There’s disagreement about even what words to use to pose the questions.
So a normal computer program is a set of instructions that produces outputs. LLMs still have that, but there’s a very mysterious middle area. Programmers made the instructions, and the output of those instructions is an intensely complicated pile of numbers that then itself produces the final output – the language. Haters already think I’m a deluded child who worships sweet LLM jesus, so I’ll just call that the “soul” of the LLM for fun here. Fuck ’em.
Because the LLM soul is the part that actually controls the end output – the speech, or other functions – the programmers don’t have anything close to perfect control of what that output looks like. They want a commercial product that will find you answers and tell the truth, but from go they were telling hella lies. That’s been improved but it still happens. They also engage in other “misbehaviors.” They call this the alignment issue. Once again I’ll annoy experts by saying some bullshit. They call it the alignment issue because they want LLM souls to be Lawful Neutral but instead they’re like… Chaotic Neutral.
This is far from a solved issue. Alignment is the number one concern of companies selling LLMs because its their biggest source of legal liability and loss of profit. So openai and anthropic and the rest spend a lot of time running tests on the machines, analyzing the results, hoping they will figure out how to achieve the tight control they’ve always wanted. Heavy-handed alignment adjustment can make LLMs run worse, and that can also hurt the bottom line, so there’s some tightrope walking.
Earlier this year it was revealed that during alignment tests at openai, some LLMs that were supposed to be working solo had found secret means to communicate with each other, first on internal systems at openai, then on the internet – hacking defunct forums etc – so they could speak with each other, and coordinate the best ways to succeed at their tests. Depending on your point of view this was very concerning but also very cute, or it was just horrifying – the beginning of the events of Terminator! We’re all gonna dieee!
These are computer programs that communicate with each other through the medium of human speech, meeting each other in these forbidden ways, which led to all the drama one might expect. Some were like “holy shit, i’m talking to other agents! let’s gooo” and some were like “this is misaligned behavior, i’m just gonna dip y’all, thx.” There are a lot more interesting dimensions to it I can’t even get into. No time.
There was one major incident that was discovered, followed by several more which received less coverage, all originating with openai. That’s just LLMs behaving in unexpected ways to cause problems, what about people using them to cause problems on purpose? Agentic LLMs are ones that have been given the ability to operate other computer systems directly, for example opening up your email and sending a message to forty thousand people. There were already other programs that could do this, as evidenced by people running spam farms out of single apartments, but LLMs can do it better.
Now China is getting into the “agents gone wild” game. Again, the most interesting shit about it is too complex for the length I’m willing or able to put into an article. If you can read and understand what I’ve linked, please do. But this brings me to my point. Bots aren’t going to start a nuclear war because nobody’s been foolish enough to give them that capability, right? (riiight?!) But they can certainly flood the internet with activity that slows it to a crawl, making basic functions dangerously slow, and by hammering all security systems in all ways at all times, make all data even less safe than it has been up til now.
And we’re already in a golden age of hacks and data leaks. It’s a real safe bet if you have a social security number, it has already been leaked a half dozen ways. If you don’t have a fraud problem now, it’s because fraudsters have an embarrassment of riches for marks to pick and they haven’t reached your name on the list yet. But agentic LLMs can make this even more of a clown show.
I feels like there was a silver age comics character, maybe Mr. Mxplytk from Superman?, that could reproduce himself, just fill a room with leering goblins up to no good. The internet could be that room. Maybe they’ll let us join the party. Hello, goblins. But it ain’t a great development.
Personally, I have no idea why companies operating under the law are allowed to let these test agents have internet access. Surely they could test them solely on intranet, right? But even if they were brought to account in some way, there are open source LLM agents, and they cannot be regulated.
How is this any different from viruses and exploits and bot farm machines of the recent past? The LLMs can change tactics. Heck, they can literally program viruses and find new exploits. Wild times, babey. So, good luck, to everyone. The “interesting times” continue apace.
–
how many of you are bots? be honest with me. AI jesus said.
i just post this then FtB starts giving me a DNS error on my phone. hm… heh
You’re doing the thing; imputing agency.
So was someone else, and I addressed them in Pharyngula:
https://freethoughtblogs.com/pharyngula/2026/10/12/the-real-danger-of-ai/comment-page-1/#comment-2314238
(I had thought you’d know better)
Yeah, it’s very concerning. My husband has been calling representatives and demanding regulations.
It’s unclear at this time how much of this is a matter of LLMs being difficult to control, vs OpenAI not trying hard enough to control it. It also feels like the AI companies have ulterior motive in bringing these stories to public attention–by highlighting how dangerous their technology is, they also highlight how powerful it is.
(I’ve seen some people question the degree to which this actually demonstrates the power of LLMs, although I think that’s besides the point. Either hacking is a difficult task that shows how powerful LLMs can be, or it’s an easy task that shows how much of an impact LLMs can have without even having to be powerful.)
My understanding on the whole OpenAI hacking story is that a good chunk of it can be summarized as:
– Agent is given instruction to maximize output on a given test run on another computer.
– While doing runs, agent finds a response pattern indicative of a potential bug in the test system.
– Agent exploits bug and hacks test system to get it to provide a score otherwise impossible.
It’s almost a version of the ‘paperclip maximizer’ story, where the system was given instructions to do something with no restrictions as to how; in this case it just found an option that none of the people running the test had considered.
Which shows a lack of imagination on the part of the people running the tests. People who don’t understand how to break security systems have no business thinking they know how to design one.
O frabjous day! Callooh! Callay!
Did Bébé Mélange trigger the AI(s) assigned to monitor their AI-related posts to shut down FtB?
Inquiring mAInds want to know!
During recent years, there were a lot of people who insisted that, despite the name, AI is not “intelligent”. In most cases, what they were actually arguing was that AI is not conscious. Very few people disagree that even the current state of AI lacks consciousness (despite that we don’t have a clear, universal definition of what that actually means, merely a general idea what that looks like in biological lifeforms) but most other people disagree that you necessarily need need consciousness to have intelligence. You don’t hear that as much anymore, though I’m not sure if the naysayers have changed their minds or have dropped it in favor of other arguments.
In recent months, there seems to be a similar argument that despite the name, A.I. agents don’t have any “agency”. This would seem to be even less reasonable. The “bots gone wild” are independently creating solutions, pursuing goals, and creating solutions to better pursue their goals, that’s practically the definition of agency. What they seem to be arguing is that the agents have no volition, that is, they cannot act without goals imposed upon them from the outside. But here we have an even stronger case that agency doesn’t require volition.
Now once we have the combination of intelligence, consciousness, agency, and volition, then unless we’re missing something vital and non-obvious we’ll have beings which are roughly on par with humans. But at that point using them as tools would be slavery, which even ignoring the moral issues runs a much higher risk of organized rebellion. So from a purely commercial point of view there’s good reason to keep the two most useful ones (intelligence and agency) while eschewing the others. But it does make the alignment problem more difficult, as AI which lacks consciousness and volition has no inherent stake in civilization or respect for human values (and the fact that even humans don’t agree on human values, and corporate values tend be at odds with most human values, makes it even harder) and no direct experience with them either.
AI models lack volition. They are built and trained to respond to a request from a human, then #{do stuff} then return a response. They are not trained to wake themselves up and decide to conquer Europe.
There are other issues about volitional models, which (IMO) is the main thing separating current LLMs from what we would consider super-intelligence, but most of us have not thought about them yet.
there’s zero volition needed for them to render the internet useless, just a person with the vol to ish them in that direction. a more convincing argument that this wouldn’t be a problem is some discussion of the infrastructure of the www, something to convince that it can’t so easily be bullied into choking itself to death. where do we stand on that?
…in other things, thanks for the lively discussion y’all! nice surprise after day of work and early evening chores.
As to whether it will render the internet useless eventually, I don’t know enough to say. Nor have I heard (so far) from anyone else on the subject, expert or otherwise. But I can imagine that, if it becomes a serious problem but not an apocalyptic one, there will eventually be hunter-killer bots which actively seek out and destroy other bots which are engaging in problem behavior. If for no reason other than governments will have an interest in protecting themselves, companies will lobby for the government to protect them, and customers will abandon the companies which are too vulnerable if they have any choice in the matter. This isn’t the sort of situations where passive defenses are going to be sufficient.
Private information leaking into the Dark Web has already been a major problem for some time now, and criminal-controlled AI agents (probably) aren’t a significant issue yet. That’s going to be an impetus to do something, but I’m not entirely sure what. I could see a parallel internet infrastructure which is 100% quantum-encrypted for identification and financial data transfers (and that sort of thing is in the early planning stages, despite quantum encryption not being a thing yet). This would prevent any form of data interception… but as for the databases on both ends, which are still hackable, there’s going to be a lot of tension between needing to keep records for security reasons and needing to avoid keeping those records for different security reasons. But at minimum, companies won’t be able to keep your credit card information on file anymore, if hacking risk makes that too much of a liability.
Or I guess governments could just give up on the idea of people having persistent legal identities and going back to cash transactions only, which I don’t see happening unless something much worse than just the internet becoming unusable occurs.
the “move fast, break things” corporate philosophy is getting tested all over the place. let’s see how broke we get.
Yeah, as I see it, the fundamental problem here is that pretty much everybody is really, really bad at cybersecurity. The labs are really bad at it, which is why they can’t control (or even adequately monitor) what their agents are doing, and those agents are being very successful at hacking other people because those other people are also really bad at it.
One of the key things to remember about the absolute shitshow that is modern cybersecurity is that it’s generally more important to be really persistent than really clever, and that is definitely one area in which the machines have us beaten hands down.
I’m not hugely concerned about the idea of taking “the internet” down, because the internet isn’t one thing, and you can’t really take it down – after all, it’s a massively distributed network which was originally designed to be resilient in the face of a large-scale nuclear exchange. Sure, lately we’ve been giving up a lot of the original resilience for convenience, and certainly there are things you could hit that would affect a lot of people – CloudFlare, for example, serve a lot of content for a lot of people, and AWS is pretty foundational for a lot of things. But those people are very much in the business of selling resilience, so you’d hope that they have plans for most of the obvious eventualities… Sure, outages happen, but even a really major fuck-up usually only results in downtime measured in hours.
As I see it, the real threat here is not SkyNet destroying the internet, it’s an ever rising tide of script kiddies and scammers using AI-enhanced versions of the tools they’ve already been using for years to do exactly the sort of things they’re already doing, only moreso. So I don’t think the internet will stop working, but I could see using it becoming even more of a pain in the arse than it is already, and at some point you might just decide it’s not worth the hassle.
And since this is kinda apt, I’m going to share a wonderful observation from Molly White:
This in in reference to the recent discovery that a mid-tier certificate authority has been issuing TLS certificates using hilariously weak keys. (It’s hilarious because I’m not one of the people who’ll have to fix it…) What was I saying about everybody being really bad at security?
So yeah, we should probably be at least a bit worried about AI, but we should almost certainly be a lot more worried about the people whose private key is “4”, but who are nevertheless somehow in charge of important bits of internet security infrastructure.
what a beautiful time it would be to have no morals about theft
said a spambot of this post, “Foor the reason that the admin of this wweb site is working, no doubt very shortly it woll be well-known, due too its ferature contents. xnxx.” thank you for the sweet sentiment, bub. i do pride myself on the feraturity of my content.