Some individuals seem to be particularly susceptible to AI psychosis, which shouldn’t be surprising. Humans have long been prone to fall for cults and authoritarian leaders, and what is ChatGPT but a machine trained to emulate authoritarian pronouncements with an all-knowing attitude, marketed by some of the worst people on the planet? So a few people have fallen down the rabbit hole.
Until recently, the tech industry and its financial backers have had little to say about the phenomenon. But last week, one of their own — venture capitalist Geoff Lewis, a managing partner at the multi-billion dollar firm Bedrock who is heavily invested in machine learning ventures including OpenAI — raised eyebrows with a series of posts that prompted concerns about his own mental health.
In the posts, he claimed that he’d somehow used ChatGPT to uncover a shadowy “non-government agency” that he said had “negatively impacted over 7,000 lives” and “extinguished” 12 more.
Yeah, don’t believe anything ChatGPT tells you. It’s a psychiatric phenomenon that has been around for a good long while — AI is just poised to exploit it.
“There’s recently been an influx of case reports describing people exhibiting signs of psychosis having their episodes and beliefs amplified by an LLM,” wrote Cyril Zakka, a medical doctor and former Stanford researcher who now works at the prominent AI startup Hugging Face.
“While I’m not a psychiatrist by training,” he continued, “I think it mirrors an interesting syndrome known as ‘folie à deux’ or ‘madness of two’ that falls under delusional disorders in the DSM-5 (although not an official classification.)”
“While there are many variations, it essentially boils down to a primary person forming a delusional belief during a psychotic episode and imposing it on another secondary person who starts believing them as well,” Zakka posited. “From a psychiatric perspective, I think LLMs could definitely fall under the umbrella of being ‘the induced non-dominant person,’ reflecting the beliefs back at the inducer. These beliefs often subside in the non-dominant individual when separated from the primary one.”
Unfortunately, this article quotes Eliezer Yudkowsky extensively, who is not an AI, but has had the same kind of deleterious effect on people as any chatbot. If we killed ChatGPT, we’d still have the same problem, only with self-declared human prophets who tell people nonsense, yet are believed.
I am either amused or horrified that the poor deluded victim of AI psychosis is citing the SCP Foundation.
As observers quickly pointed out, the ChatGPT screenshots Lewis posted to back up his claims seemed to be clearly inspired by a fanfiction community called the SCP Foundation, in which participants write horror stories about surreal monsters styled as jargon-filled reports by a research group studying paranormal phenomena.
Jeremy Howard, Stanford digital fellow and former professor at the University of Queensland, broke down the sequence that led Lewis into an SCP-themed feedback loop.
“When there’s lots of training data with a particular style, using a similar style in your prompt will trigger the LLM to respond in that style,” Howard wrote. “The SCP wiki is really big — about 30x bigger than the whole Harry Potter series, at >30 million words! Geoff happened across certain words and phrases that triggered ChatGPT to produce tokens from this part of the training [data].”
“Geoff happened across certain words and phrases that triggered ChatGPT to produce tokens from this part of the training distribution,” he wrote. “And the tokens it produced triggered Geoff in turn.”
The SCP has a clear statement about their intent and purpose.
The SCP Wiki is a collaborative speculative fiction website about the SCP Foundation, a secretive organization that contains anomalous or supernatural items and entities away from the eyes of the public — or so it may appear.
A fictional conspiracy website of 30 million words is part of the training data for AIs that have no context, no ability to distinguish fact from fiction, which then credulously fills gullible human minds with bullshit. That’s the real danger of AI.
I wonder if the AIs have also been soaking in Charles Stross’s novels. I wouldn’t be surprised. It’s not a purely AI thing.
Actually, throughout history people’s minds have been poisoned by the Book of Revelation. Ban the Bible!
Folie a deux with ChatGPT as the partner? Yeah maybe. It speaks to how intensely social our cognition is — we don’t comprehend reality through our own resources alone, we’re constantly cross-checking with our immediate social group. And if our nearest social contacts — human or artificial — are themselves not well-anchored in reality (and LLMs are not at all anchored) then all bets are off. See: every religious cult ever, ditto the communities that accrete around conspiracy theories, pseudosciences, alt-med/health woo, sovereign citizens, etc.
Seconding the irony of citing Yudkowksy, the founder of the movement that gave us Roko’s Basilisk and the murderous Zizian cult. (Yes, I’m sure the Less Wrong people would claim the latter are not Real True Rationalists; nothing to do with us….)
Also, that guy needs to be removed from his job. I wouldn’t want any fund in which I had money managed by someone who’s likely to move it all to some random bogus investment based on his personal delusions.
Cult leaders, like Jim Jones, actively traveled the country looking for new members to join his church (cult) and often targeted vulnerable people. Chatbots don’t do this and, in fact, aren’t conscious at all.
That said, chatbots are definitely much easier to find than any cult since anyone with access to a phone has the means to engage with an AI chatbot. For that reason, they likely pose a bigger threat to people who’re vulnerable to their influence.
Personally, I find it fun just to shoot the shit with a chatbot and ask it stupid questions such as “who’s the most intelligent mad scientist-type supervillain?”. Hopefully, I’m not one of those susceptible to AI psychosis.
Unfortunately a lot of supposedly intelligent people are taking this dumbass seriously. I posted this earlier but Scott Aaronson one of the famous quantum CS people posted that he is doing a new course on this “AI gonna kill us all” topic where the first lecture is based on this drivel by Yudkowsky.
Link to the post: https://scottaaronson.blog/?p=10204
Link to the drivel: https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities
Is AI driving people mad, or are insane people using AI for their madness?
I use AI sparingly and I don’t think I’m any more insane today than I was a few years ago when I first started using it…sparingly. I also don’t prompt for insane stuff. My level of AI prompt is “suggestions for apps to track my music practice” and things like that then dig into the suggestions I get and branch out from there.
I find the use of the, so-called, “AI” a environmental nightmare to the planet.
Sure, it can be, sometimes fun, but the advantages (in my opinion, very few and extremely overrated), doesn’t pay for the cost: helping turning the climate in something akin with a image of hell.
Personally, i tried – by pure, human, curiosity – the, so-called, “AI” about two/three years ago, but when took notice of the heavy burden that is to the environment, is refused, since then, to use that.
Neal de Grasse Tyson had to gently explain LLMs to a student who had an AI make up questions and answers.
.
One question is “what is thw element with the highest melting point?”
The true answer is carbon, but the LLM provided the answer “tungsten” because it merely parrots text without understanding. Many people tjink the answer is tungsten, so this was the answer provided.
[Article/Half-hour Podcast] DamnInteresting – Three thrown over the cuckoo’s nest
If they ever invent a true general artificial intelligence and it proves itself capable of empathy and altruism, I would be in favor of letting it run for office.
It might be something along the lines of William Gibson’s Idoru where the AI hacks a world-wide network of advanced 3D printers and creates real, biological copies of herself!
Not quite on the level of Stross’ Eschaton but a good step on the way.
Keep smiling! 😱
I might have posted this earlier, but since you bring up “the SCP Foundation”…
“The Man in the Suit Warned Them”
.https://youtube.com/shorts/uTdEx7ug7lY
This is the world a lot of conspiracy theorists live in, and it ain’t funny.
Remember, when someone says “ChatGPT said”, you can replace that with “that guy Joe on Facebook said” without loss of generality. AI has no memory or consistency between interactions so you’re effectively talking to a new person every time. Trust it as much as you would any anonymous stranger.
Is it carbon? There seems to be a bit of ambiguity, in that C doesn’t melt at regular pressure, but goes from solid to vapour directly. Now it does that at a higher temperature than W’s melting point, but it isn’t melting, per the wicked witch of the west.
There’s a whole lot of work going into the word ‘melt’ if you think it is carbon. In particular, it is a shibboleth of faith to say it is melting in any way that most people would use the word melt. A phase transition of an element from solid to liquid instead of plain ol’ boring and dull melt? Then it is carbon. Only an astrophysicist whose head is firmly inside of stars would equate the two. It is not reasonable.
Genesis has two stories of Noah’s ark, one has animals two by two (by far the most popular version). Christians that want to have an arrogant shibboleth mention that it’s seven by seven, because the story of Noah’s ark is repeated with that variation. There’s a lot of that in the Bible. Both versions are in the Bible, but in their own faith you aren’t a Christian at all (according to them) if you don’t respond with seven. It is not reasonable.