Some individuals seem to be particularly susceptible to AI psychosis, which shouldn’t be surprising. Humans have long been prone to fall for cults and authoritarian leaders, and what is ChatGPT but a machine trained to emulate authoritarian pronouncements with an all-knowing attitude, marketed by some of the worst people on the planet? So a few people have fallen down the rabbit hole.
Until recently, the tech industry and its financial backers have had little to say about the phenomenon. But last week, one of their own — venture capitalist Geoff Lewis, a managing partner at the multi-billion dollar firm Bedrock who is heavily invested in machine learning ventures including OpenAI — raised eyebrows with a series of posts that prompted concerns about his own mental health.
In the posts, he claimed that he’d somehow used ChatGPT to uncover a shadowy “non-government agency” that he said had “negatively impacted over 7,000 lives” and “extinguished” 12 more.
Yeah, don’t believe anything ChatGPT tells you. It’s a psychiatric phenomenon that has been around for a good long while — AI is just poised to exploit it.
“There’s recently been an influx of case reports describing people exhibiting signs of psychosis having their episodes and beliefs amplified by an LLM,” wrote Cyril Zakka, a medical doctor and former Stanford researcher who now works at the prominent AI startup Hugging Face.
“While I’m not a psychiatrist by training,” he continued, “I think it mirrors an interesting syndrome known as ‘folie à deux’ or ‘madness of two’ that falls under delusional disorders in the DSM-5 (although not an official classification.)”
“While there are many variations, it essentially boils down to a primary person forming a delusional belief during a psychotic episode and imposing it on another secondary person who starts believing them as well,” Zakka posited. “From a psychiatric perspective, I think LLMs could definitely fall under the umbrella of being ‘the induced non-dominant person,’ reflecting the beliefs back at the inducer. These beliefs often subside in the non-dominant individual when separated from the primary one.”
Unfortunately, this article quotes Eliezer Yudkowsky extensively, who is not an AI, but has had the same kind of deleterious effect on people as any chatbot. If we killed ChatGPT, we’d still have the same problem, only with self-declared human prophets who tell people nonsense, yet are believed.
I am either amused or horrified that the poor deluded victim of AI psychosis is citing the SCP Foundation.
As observers quickly pointed out, the ChatGPT screenshots Lewis posted to back up his claims seemed to be clearly inspired by a fanfiction community called the SCP Foundation, in which participants write horror stories about surreal monsters styled as jargon-filled reports by a research group studying paranormal phenomena.
Jeremy Howard, Stanford digital fellow and former professor at the University of Queensland, broke down the sequence that led Lewis into an SCP-themed feedback loop.
“When there’s lots of training data with a particular style, using a similar style in your prompt will trigger the LLM to respond in that style,” Howard wrote. “The SCP wiki is really big — about 30x bigger than the whole Harry Potter series, at >30 million words! Geoff happened across certain words and phrases that triggered ChatGPT to produce tokens from this part of the training [data].”
“Geoff happened across certain words and phrases that triggered ChatGPT to produce tokens from this part of the training distribution,” he wrote. “And the tokens it produced triggered Geoff in turn.”
The SCP has a clear statement about their intent and purpose.
The SCP Wiki is a collaborative speculative fiction website about the SCP Foundation, a secretive organization that contains anomalous or supernatural items and entities away from the eyes of the public — or so it may appear.
A fictional conspiracy website of 30 million words is part of the training data for AIs that have no context, no ability to distinguish fact from fiction, which then credulously fills gullible human minds with bullshit. That’s the real danger of AI.
I wonder if the AIs have also been soaking in Charles Stross’s novels. I wouldn’t be surprised. It’s not a purely AI thing.
Actually, throughout history people’s minds have been poisoned by the Book of Revelation. Ban the Bible!
Folie a deux with ChatGPT as the partner? Yeah maybe. It speaks to how intensely social our cognition is — we don’t comprehend reality through our own resources alone, we’re constantly cross-checking with our immediate social group. And if our nearest social contacts — human or artificial — are themselves not well-anchored in reality (and LLMs are not at all anchored) then all bets are off. See: every religious cult ever, ditto the communities that accrete around conspiracy theories, pseudosciences, alt-med/health woo, sovereign citizens, etc.
Seconding the irony of citing Yudkowksy, the founder of the movement that gave us Roko’s Basilisk and the murderous Zizian cult. (Yes, I’m sure the Less Wrong people would claim the latter are not Real True Rationalists; nothing to do with us….)
Also, that guy needs to be removed from his job. I wouldn’t want any fund in which I had money managed by someone who’s likely to move it all to some random bogus investment based on his personal delusions.
Cult leaders, like Jim Jones, actively traveled the country looking for new members to join his church (cult) and often targeted vulnerable people. Chatbots don’t do this and, in fact, aren’t conscious at all.
That said, chatbots are definitely much easier to find than any cult since anyone with access to a phone has the means to engage with an AI chatbot. For that reason, they likely pose a bigger threat to people who’re vulnerable to their influence.
Personally, I find it fun just to shoot the shit with a chatbot and ask it stupid questions such as “who’s the most intelligent mad scientist-type supervillain?”. Hopefully, I’m not one of those susceptible to AI psychosis.
Unfortunately a lot of supposedly intelligent people are taking this dumbass seriously. I posted this earlier but Scott Aaronson one of the famous quantum CS people posted that he is doing a new course on this “AI gonna kill us all” topic where the first lecture is based on this drivel by Yudkowsky.
Link to the post: https://scottaaronson.blog/?p=10204
Link to the drivel: https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities
Is AI driving people mad, or are insane people using AI for their madness?
I use AI sparingly and I don’t think I’m any more insane today than I was a few years ago when I first started using it…sparingly. I also don’t prompt for insane stuff. My level of AI prompt is “suggestions for apps to track my music practice” and things like that then dig into the suggestions I get and branch out from there.
I find the use of the, so-called, “AI” a environmental nightmare to the planet.
Sure, it can be, sometimes fun, but the advantages (in my opinion, very few and extremely overrated), doesn’t pay for the cost: helping turning the climate in something akin with a image of hell.
Personally, i tried – by pure, human, curiosity – the, so-called, “AI” about two/three years ago, but when took notice of the heavy burden that is to the environment, is refused, since then, to use that.
Neal de Grasse Tyson had to gently explain LLMs to a student who had an AI make up questions and answers.
.
One question is “what is thw element with the highest melting point?”
The true answer is carbon, but the LLM provided the answer “tungsten” because it merely parrots text without understanding. Many people tjink the answer is tungsten, so this was the answer provided.
[Article/Half-hour Podcast] DamnInteresting – Three thrown over the cuckoo’s nest