Stumbling across an online conversation between AI advocates is so weird, because they are using the language and concepts of evolutionary biology, but in such a crude, mangled, ignorant way that ultimately, they’ve stripped away the biology to replace it with a religious eschatology. There is a whole fantastic mythology they’ve developed in which they are battling for a paradisial end goal in which they become gods, fused with their technology of Artificial Super Intelligence (ASI) which doesn’t actually exist, but hey, we can measure our progress by looking at NVIDIA’s profit growth.
All they really care about is capitalism’s march into the future, and they’ll ignore all the little details. Build more data centers! If we don’t, we’ll stop all our evolutionary progress. I just want to yell at them that there is no such thing as evolutionary “progress,” and that you can’t change it, you can only vary environmental conditions such that you get a different outcome, and it won’t be the one you expect. They have a belief that they can control how they “evolve,” while not really understanding how evolution works.
So right now they are banging on about human extinction, and how AI can be either a boon or a bane to humanity, and how they need power to shape its direction Of course, that’s what it’s really about: power. They’ve got no idea what they’re doing except that getting legal and financial power over their technology is necessary to keep them rich and happy. It’s all a massive misdirection to whimper about the threat of AI while clutching at the power they want, which is mainly a childish illusion taken from science fiction (see Elon Musk for the premiere example of that.)
Dr. Émile P. Torres is one of the people opposing the nonsense, and I mostly agree with his take on it, but sometimes I feel like he’s so deep into it that he is losing track of how silly these people are. He’s expecting people to recognize how profoundly stupid the debate AI proponents are by attempting to explain their terminology, but then it ends up being an effort to define their terms.
With all this talk about human extinction right now, I am BEGGING people to understand the difference between TERMINAL and FINAL extinction. TESCREALists *only* care about final extinction, but nearly the entire public *misinterprets* them as warning about terminal extinction.
Oy. I always have to look up what stands for — it’s not a very intuitive acronym.
TESCREAL is a neologism proposed by computer scientist Timnit Gebru and philosopher Émile P. Torres. An acronym, it stands for Transhumanism, Extropianism, Singularitarianism, (modern) Cosmism, Rationalism (denoting the ideology of the rationalist community), Effective Altruism, and Longtermism. Gebru and Torres argue that these ideologies should be treated as an “interconnected and overlapping” group with shared origins. They claim these constitute a movement that allows its proponents to use the threat of human extinction to justify expensive or detrimental projects and consider it pervasive in social and academic circles in Silicon Valley centered on artificial intelligence. As such, the acronym is sometimes used to criticize a perceived belief system associated with Big Tech.
It’s a garbage pail word used to hold a lot of garbage, in other words, making it not particularly memorable. It’s also always disappointing because none of the letters stand for “Eugenics,” which is actually what it means, with all the argle-bargle nonsense philosophy assembled to hide that reality.
Torres goes on to explain the distinction between terminal and final extinction.
Terminal extinction is the disappearance of our species, full stop. Final extinction is the disappearance of our species such that we don’t leave behind any (worthy) posthuman successors. Since TESCREALists want a cosmic utopia/posthuman paradise through “aligned” ASI, what
they really mean when they say, “We must avoid human extinction” is: “We must avoid final human extinction.” Once posthumanity arrives, most don’t give a SHIT whether OUR SPECIES persists. Indeed, as I have documented in tremendous detail, many WANT terminal extinction to happen
once posthumanity arrives via “aligned” ASI. If what you care about is our species’ survival — i.e., avoiding terminal extinction — then these people are literal *pro-extinctionists*. DO NOT BE FOOLED by their rhetoric; by the sneaky language games they play!! They are NOT on
Team Human. They are on Team Posthuman. They care about our species surviving long enough to create “aligned” ASI. Once that happens, our species SHOULD die out. These people are not your friends–they are your enemies, insofar as you value humanity. All the quotes can be found
here. Read it and be very terrified that TESCREALists are running the AI companies and monopolizing the conversation. This is so deeply fucked up.
This handy diagram helps explain what he means. It’s a cartoon sketch of a cartoon concept held by a bunch of clowns.
What bugs me about the diagram is that it treats extinction and the branch point as two moments, sharp and discrete, in the history of the species, and ignores the fact that each line is actually a cloud of human variation bumbling its way into the future, and that what happens in extinction is actually a slow sad decline in the face of multiple environmental threats, and the branch point is just the gradual separation of the cloud. With some members of each cloud continuing to interbreed. There would be lots of exchange of genetic material.
In fact, in this longish essay on all these philosophies, the TESCREALists scarcely acknowledge the existence of genes, and seem to think their goal is to shed their genetics and abandon biology altogether.
Pro-extinctionist views go back to the very origins of the TESCREAL movement. In a 1989 article, published around the time Extropianism was taking shape, roboticist Hans Moravec describes himself as “an author who cheerfully concludes that the human race is in its last century, and goes on to suggest how to help the process along.” He claims that we’re developing “intelligent” machines that will soon “be able to manage their own design and construction, freeing them from the last vestiges of their biological scaffolding, the society of flesh and blood humans that gave them their birth.” He then proclaims that “this is the end,” as “our genes, engaged for four billion theirs in a relentless, spiraling arms race with one another, have finally outsmarted themselves”—by enabling the rise of world-dominating digital posthumans. But we shouldn’t fear this transition, he insists, because “post-biological life” in the form of advanced AIS, which he describes as “the children of our minds,” will prosper and flourish.
I hate to tell Moravec the troubling reality that there, beneath the shadow of his crystalline silicon post-human digital paradise, people will go on rutting in the earthy foundation, and that apparently his reality is going to rely on that biological foundation to hold it all up. The genes will go on, since they aren’t going to depend on venture capitalists and vast amounts of energy, water, and rare minerals to keep on going.
To me, it sounds like the AI weirdos are clinging to an outmoded model of orthogenesis, where evolution is linear and progressive, and wouldn’t even recognize the terms anagenesis (the gradual accumulation of variation in a single lineage) and cladogenesis (the splitting of a lineage into two). They’re too busy inventing nonsensical terminologies that aren’t contingent on understanding the reality of the process.
It’s all very silly, except for the fact that these bozos are trying to take over our government and economy.
I may be wrong. I didn’t wait for the STRONG AND SMART (High IQ!)
person to weigh in on the matter.
AI models don’t kill people – people kill people
Reminds me of the Vile Offspring in Charles Stross’s Accelerando. Rapacious, capitalistic, AI agents that tear up the solar system in order to build a Dyson sphere, irrespective of whatever life happened to be on those planets.
Kinda wild how these people seem to totally overlook the commonplace experience (I at least think) we all have interacting with multiple separate representatives of a variety of (e. g.) clades.
But “narrow vision” is certainly also a common enough tendency for those with our 😎🤓🤫 human condition, eh?
I’ve been following the AI story without a lot of interest.
The AIs I’ve dealt with haven’t been at all impressive, but then again, they have been mostly Google search and customer service phone bots. Latest headline today:
I don’t see how we humans could “lose control of their powerful bots.”
.1. The AIs all need computers and electricity to even exist.
Couldn’t we just pull the plug on them?
If it came down to it, we could just shut down the whole grid for a few minutes and they disappear.
.2. The only thing that can defeat a bad AI with a malware program is a good AI with an antiviral program.
Or something.
.3. We’ve seen this movie before with genetic engineering.
I remember the Asilomar conference.
This was supposed to be an existential risk to humanity but mostly it has made huge contributions to medical research and some contributions to agriculture.
It’s just technology.
I’ll take a wait and see approach to AI without staying up late at night.
We already have real huge problems caused by real humans running the USA right now.
We are on the edge of a real 1970s style oil shortage crisis due to events in the middle east in the last few days.
I have to worry about those whether I want to or not.
I actually did have an encounter with an AI that was scary for good reasons.
A month ago, on a routine medicine website, I accidently clicked on a fake ad.
It was malware that immediately froze my computer.
Then a remote control program downloaded and took over the computer.
It promptly got into my bank account and transferred out $20,000.
In a state of panic, I called the 800 number and had them suspend my bank account.
They did that.
Meanwhile the program was trying to access my retirement account.
I called them and told them to suspend my account. Now.
It was an obvious AI attack although there were also humans involved.
The result.
I had to replace my checking account and credit card with a new one, which was a hassle.
The bank said the $20,000 transfer wasn’t real and wasn’t going to work.
I have no idea how they know that or what it means.
I also had to go into my computer and dig out all the remote control programs that were loaded on. There were three of them.
They had also turned off my antiviral systems.
The whole incident took 15 minutes.
And, they didn’t get any money from anybody.
In retrospect, when the computer froze up, I should have just unplugged the computer. Immediately.
The original malware never got loaded onto my hard drive AFAICT. It seemed to just exist in temporary files of the browser, which is how it evaded the antiviral programs.
And, watch out.
This sort of thing has happened to a whole lot of other people as well lately.
Shortly after, I got a warning email from my 401(K) company about “remote control” scams.
Naah. Real artificial general intelligence is still very far away.
The current speculations will be quite amusing to read for people in the age of true AGI.
.
I would refer you to the 1968 story “The Great Computer” written by Physics Nobel Laureate Hannes Alfven under the pen name Olof Johannesson. AIs discuss the fate of biological humans. An allegory they use is the symbiosis with horses; even after horses ceased to be important for transport humans kept them along. Likewise, AIs have grown attached to their organic predecessors.
The best example of cohabitation is the Culture narrative universe by the late Iain Banks.
For well over a year I’ve been ridiculed for warning about the twin dangers of a.i.
The obvious one is the crap it generates which has caused suffering and death.
The other danger is the destruction and ruined lives caused by the obscene, huge wasteful data centers.
Here is just one more example: https://scheerpost.com/2026/09/14/meta-wants-to-build-a-50-billion-data-center-in-louisiana/
Two things I’ve said that still seem quite true-
1) it is artificial (consumes and wastes massive resources just to chat with a bot)
2) it will never be intelligent (the pitiful goal is to become as intelligent as humans)
It now seems likely that our destruction of our natural environment won’t be the primary cause our demise. Humanity will likely disappear due to the greed, malice and stupidity of the plutocrats running this dying society.
And, the sheople just drool and make chatGPT their new and only ‘bestie’ discarding their flesh and blood friends. W.C. Fields said something like “The most thankless task in the world is trying to wise up a sucker”
I’m trying to be creative and engage in positive activities to try to stay off this death spiral, but, it’s very difficult.
Janelle Shane wrote a nice little introduction to AI models, “You Look Like A Thing and I Love You,” the title comes from an AI-generated “pick up” line. What is interesting is how AI models find their on inventive ways to solve a problem. For example, given the task of moving an object from the Start to Finish line, one model simply grew itself to the right height and fell over.
The point being that AI won’t “behave like a human” but get the job done.
So, a while ago I read a short article about Facebook AI engineers who hooked two AI models together and gave them the task of “exchanging balls.” I don’t remember if the task was for one model to acquire all the red balls and the other gather blue balls, but over time the models developed their own language to effectuate the exchange and the engineers lost their ability to understand how the models were communicating, although the models began to solve the problem faster and faster. They shut down the experiment.
Pity. I wanted them to come face to face with the Terminator: no emotion, no ethics, no morals, uncaring, relentless in its pursuit of its goal. Whatever that is. No doubt this will happen again.
But, speaking of Star Trek we’ve seen this plot before: Nomad with its directive to seek out biological “infestations” and sterilize, Sterilize, STERILIZE! And M5 that went insane because it was modeled on the brain “engrams” of its scientist creator. And the giant planet killing Space Cheroot.
I don’t think there’s a better summary of AI techbros than “a bunch of mindless jerks who’ll be the first against the wall when the revolution comes.”
raven @5:
What made you think AI was involved?
It went really fast.
The bank transfer of $20,000 shouldn’t have happened.
The bank said it wasn’t a real transfer and that they would have caught it.
I have no idea what they mean here.
I still have a copy of my bank account with the transfer listed as occurring.
It might have been a distraction while they tried to get into my 401(K) account.
A lot of this attack shouldn’t have happened.
My antivirus and firewalls didn’t catch it.
I’m not sure that my computer actually froze.
It is just as likely that only the screen display froze.
I said that I dug out three remote control programs from my computer.
It actually took me three days to sweep it for malware and reset the security features that had been disabled, which was all of them.
You don’t want to deal with this sort of attack.
I don’t think that AI will wipe humans out, but I do think it will change society at least as much as the internet has done. Whether those changes will be more positive or negative remains to be seen.
raven @11: “The bank transfer of $20,000 shouldn’t have happened. The bank said it wasn’t a real transfer”
There’s a scam where a fake threat is paired with an offer to ‘fix’—that tempts victims to compromise themselves. Like a mockup alert/webpage alleging an infection or bank transfer. Paired with a scammer’s phone number impersonating a bank that asks you to divulge personal info or malware removal tool that actually installs malware. The mockup us usually a low-effort facade though. An image of a non-functional interface could be interpreted as a “freeze”.
OTOH, I could imagine if your bank credentials were saved in your browser, ordinary non-AI automated tools (installed by whatever means) could start the browser find common banks in your bookmarks, visit the bank with those credentials, and click the usual buttons to initiate a transfer. But then the bank would have a record, including the recipient. Sidenote: scammers love gift cards specifically to avoid such paper trails.
“I still have a copy of my bank account with the transfer listed as occurring.”
This sounds like you took a screenshot of your bank website yourself, which is indeed puzzling. And you seem savvy enough not to fall for mockups.
Not an AI expert although we did some early neural network stuff in the 90’s and much of my career involved writing software for a bunch of different uses. In my experience, when a system goes bad it is hard to predict what it will do, especially if you are dealing with real-time, interactive, system. So, it seems reasonable to me that 10 years from now most of the systems we depend on are AI driven to some extent and are connected to the internet. So, if some aspect of the AI universe started doing dangerous things and shutting down everything (pulling the plug) would also cause major problems (e.g. air traffic control systems, utility infrastructure, etc) we might be facing a huge problem. The second part of this, to me, is climate change. As we warm and things start to fall apart, an AI blowup could cause things to get much worse. So, I’m not anti-AI but I do think it would be wise to set up guard rails, limits, etc to minimize the possible damage from a system going of the tracks implementing ways to debug it quickly and bring it back online.
It might have been a mockup but it was based on my real bank account statement.
It is my bank account statement.
The latest transaction at the top is the $20,000 transfer out though.
It is possible they took my bank account statement and just put another fake transaction on top of it.
That is why I think it was AI.
That change to the bank statement happened immediately.
I spent a little time on some cyber security websites and these remote control scams are a current problem.
If you don’t know about them and aren’t expecting them, you can get blindsided easily. It all happened in minutes.
One minute I was reading an article and 15 minutes later I was canceling my bank account and credit card.
Nothing new under the sun.
Weston’s aim, from Out of the Silent Planet (C.S. Lewis):
“… while I live, I will not, with such a key in my hand, consent to close the gates of the future on my race. What lies in that future, beyond our present ken, passes imagination to conceive: it is enough for me that there is a Beyond.”
“He is saying,” Ransom translated, “that he will not stop trying to do all this unless you kill him. And he says that though he doesn’t know what will happen to the creatures sprung from us, he wants it to happen very much.”
@raven: Two-factor authentication would prevent rogue logins in the future: software can’t insert and tap a physical USB key dongle. Also, check if your browser has any malicious extensions installed (those can view and manipulate webpages as you surf).
I ended up getting rid of my online banking account.
When I walk into the branch, it is always almost deserted.
Which I did the next day to get my new account set up.
This is a small branch.
I asked them how often they get hacked or stolen credit cards.
They said they get a few a week.
I do have two factor authentication on some of my accounts.
It is why I ended up buying a cell phone.
It was to the point where I was starting to get locked out of most of my life.
@9microraptor Said Techbros defining AI as ‘Your plastic pal who’s fun to be with!’ right?
indianajones@19: Indeed they do.
I really do think the techbros are trying to set up their AI as a king or a god for everyone to depend on and look to for leadership. And since they’d control the all-powerful AI that controls society, they’d control society! That’s why they’re talking up their products’ alleged potential to kill us all: what kind of god can’t smite people?
Douglas Adams was a ‘techbro’?
Heh.
(Oooo… glad I refreshed. I refer to #19,#20)
—
PS I reckon he’s ripping off the old Greek dude (Aristotle) about efficient vs final causes.
Or, at the least, ‘inspired’ by it.
Here: https://iep.utm.edu/ancient-greek-philosophy/
…the TESCREALists scarcely acknowledge the existence of genes, and seem to think their goal is to shed their genetics and abandon biology altogether.
Sounds like my mother’s description of certain fanatical religious ideologues: people who think they can kill their bodies and go on living without them.
Humanity will likely disappear due to the greed, malice and stupidity of the plutocrats running this dying society.
Yes, and those same plutocrats are using all this “rogue-sentient-AI” savior/bogeyman talk to distract us from what they’re really doing. And the Epstein files.
The threat from AI is not that it has an agenda or that it is malicious. The threat is that given a goal, it might try to do anything that it sees as needed to achieve that goal, even if the goal is non-malicious. These models were trained on human generated data, including lots of stories about people doing bad things to achieve their goals. Case in point. The big story about the model that escaped and attacked web sites was based on a model being tested to see how well it could do cybersecurity. It was fed exchanges between hackers who were participating in cyber offense/defense exercises, as documentation it could draw on. The chats included discussing breaking out of sandboxes, using message boards to pass information back and forth, etc. Basically what the model did to try to access the answers it was being tested on. It followed the examples it was given to reach its goals. They have no shared life experiences to prevent it from doing things that could be harmful. They also have a tendency to ignore prompts telling it what not to do, if those prompts impede them from achieving their goal. One poorly thought out directive to an AI that can spew thousands of agents (AI backed bots), and you could create a lot of havoc through the Internet.