I’ve always thought of Nick Bostrom as a font of bad ideas, but that’s not his self-estimation. He considers himself a genius!
Bostrom considers himself to be a towering intellect of cosmic-historical importance. He is not shy when boasting about his supposed accomplishments, even quoting on his website an article that describes him as the “Swedish superbrain.” For many years, he bragged that — to quote his profile on Edge.org — “his performance as an undergraduate set a national record in Sweden,” though he backed away from this after Simon Knutsson pointed out that there is no evidence to support such a claim. (See the “Potentially Dishonest Self-Promotion” section of this article.)
The kinds of ideas he claims to have come up with are, for instance, the belief that AI represents an existential risk comparable to a nuclear holocaust which must nevertheless be pursued zealously, making him a hero to the capitalists building massive data centers right now. He also promotes the belief that potential people could be so numerous that it’s more valuable to create conditions that enable space flight and virtual people in computers than to actually help the poor and develop an egalitarian politics, again endearing him to the hyper-rich.
In brief, the paper argues that if we colonize space — especially as digital posthumans — the future population could be enormous. Bostrom estimates some 10^38 digital people within the Virgo Supercluster, a cluster of galaxy clusters that includes our own Milky Way. (That’s a 10 followed by 38 zeros, by the way.)
If one accepts a utilitarian perspective, this means the expected value of ensuring the realization of all those future people (assuming they have net-positive lives) far exceeds the value of, e.g., helping poor people today. If you’re a truly altruistic person who aims to maximize your philanthropic impact, then you should focus not on present-day people but those living perhaps millions or billions of years from now. It’s not that future digital people are inherently more valuable. It’s just a numbers-game: there are so many more of them (potentially) than present-day people, they overwhelm the simplistic equations of utilitarianism’s moral mathematics.
Bostrom’s conclusion is that we should colonize the universe as soon as possible, because every single century, decade, and even second of delaying this results in an “astronomical” loss of potential value — i.e., the non-existence of people who could have otherwise been created in some distant galaxy if only we had started colonizing just a bit sooner. However, Bostrom also concludes that “priority number one, two, three and four should … be to reduce existential risk.” Space colonization should therefore be priority number five once we secure our survival. Perhaps you can see why Musk loves this paper.
It’s an incredibly stupid idea — it also has ties to eugenics and natalism and other flawed beliefs — and I have weird feelings about this article. It’s mainly functioning to excuse Bostrom from fault. Bostrom didn’t come up with all these bad ideas! No, he just stole all of them. He doesn’t have original ideas!
Yet, because I’m so conversant with the foundational literature of the TESCREAL movement, I’ve noticed that a large number of the “original,” “novel,” even (supposedly) “groundbreaking” ideas attributed to Bostrom actually originated with other thinkers. As early as 2018, I was posting about this on social media and privately sharing examples with colleagues — some of whom were rather shocked to see Bostrom’s work juxtaposed with that of others (from whom he borrowed without sufficient attribution). I wrote to one colleague that Bostrom’s “genius” seems to be taking ideas from other theorists and repackaging them in his own language such that they’re just different enough for him to avoid being accused of intellectual plagiarism. His oeuvre reminds me of the line, often misattributed to Picasso: “Good artists copy; great artists steal.”
It’s a long and thorough article, and traces Bostrom’s soft plagiarism back to multiple sources, but one of them is notorious crackpot and self-promoter Eliezer Yudkowsky.
Shouldn’t that be instantly disqualifying?
Sure but, because tech billionaires are aligned with future people, among present-day people they are much, much more important than the rest of us. Facts are facts! Additionally, since it is a well known fact that that tech billionaires are going to live forever, a good argument can be made that tech billionaires should really be counted as future people and therefore inherently more valuable.
Half of these things are the sort of things that thousands of designated stoner friends have come up with while chilling, in a dazed & confused state so it’s hardly surprising. Like,… D00D, what if we’re living,… like in a simulation,… like a giant computer game…. That’d be so far out, D00D!
The numbers argument that there is only one in a trillion chance that there will be 10^38 people in the universe is very similar to the argument that there may be only one in a trillion chance that God exists but we will exist in eternity if he does.
And the economist destroys the numbers game with two words. “Future discount”. The numbers game of made up numbers can be defeated by a different made up number.
Yeah, this is just wrong, crackpottery, lunatic fringe stuff.
IIRC, this is the so called “effective altruism”, which is neither effective nor altruism.
Those future people are imaginary. They don’t exist.
The people living today are real and do exist.
You could easily reverse their argument.
If we don’t take care of the earth today and the people living right now, there isn’t going to be future for any humans.
@4, remyporter,
My thought exactly; the net present value of all these possible future people is zero.
For people who claim to be geniuses, they seem to have no conception of economics (or even basic math).
This reminds me of something I read recently:
“One who boasts of his own ability has no merit.” – Lao Tzu
Well then, if all these tech billionaire arseholes believe they are in the process of making huge progress for the future, and we should focus entirely on the future rather than the present, perhaps they would like to pay right now the quadrillions of pounds in tax their inevitable vast big-brain endeavours will accrue?
“Like,… D00D, what if we’re living,… like in a simulation,… like a giant computer game….”
.
The idea of sapient beings existing in a purely digital environment was to my knowledge first presented in a short story by Stanislaw Lem in the 1960s. German readers will be familiar with it from the anthology Nacht und Schimmel. I do not know the title od the Polish anthology that first published it.
Stanislaw Lem and Arthur C. Clarke between them came up with a lot of the ideas that today are staples of SF.
Nick Bostrom reminds me of another fake scholar, Steven Pinker.
They both repackage obvious ideas and then pretend they actually have something important to say.
There are a lot of them out there and PZ has mentioned a few other names.
I generally avoid them because crackpots are a waste of time when we have real people and real problems to deal at this time (the present 2026) and in the place where we all live (reality).
One of Steven Pinker’s big ideas is that things are steadily getting better.
It isn’t exactly wrong.
If you draw a line with two points, from the stone age to the present, then yeah, things have gotten better over time.
But this is trivial and obviously true.
.1. It neglects the fact that the progress of humankind is the result of hard work and hard thinking by millions of people over millennia. It didn’t just happen, humans created the modern world and a lot of them were killed in various gruesome ways to bring it about.
.2. That line with two points is also misleading.
There have been a huge number of zigzags over the millennia and a lot of times when things were actually getting worse. We are in one of those times right now in the USA.
It is quite likely that you could spend your life in one of those zigszags, where everything is steadily getting worse. It’s happening now in places like Syria, Iran, Gaza, Russia, North Korea, parts of Africa, and of course, the USA.
The Idea of human enhancement (which he is advocating elsewhere) is, however, a valid concept, especially as we identify the gene variants associated with various diseases, and also identify -from the genomes of other vertebrates – genes that help cell repair, delay ageing and fight cancer. I would not mind a healthy life lasting the span of bowhead whales.
.
Identifying the genes programming human nature would be an enormous task, but it would help if we got less prone to hostility. Our ancestors have already modified their nature for millions of years and we are certainly less violent than chimpanzees but there is more to be done…
.
Finally, if humans are ever to be cryogenically preserved – for instance, during long space journeys- this will require a lot of changes at the cellular level, otherwise what is thawed out will be dead mush.
I’m sure Chief Justice Roberts would be happy to join with this Bostrum guy in declaring all those
imaginary“potential lives” to be fully real and deserving of full Constitutional rights.…the paper argues that if we colonize space — especially as digital posthumans — the future population could be enormous…
Except that “we” the humans reading or writing that article are not, and never will be, the “we” who might colonize space “as digital posthumans.”
Stanislaw Lem and Arthur C. Clarke between them came up with a lot of the ideas that today are staples of SF.
Also Ferrett Steinmetz dealt with all this “digital post human” shit far more honestly in “The Uploaded.” Also check out the anime miniseries “Pantheon” on Netflix, which is based on short stories by Ken Liu. If any of this “digital posthuman” stuff ever becomes more than ridiculous vaporware, it won’t be anywhere near as wunnerful as these stupid broligarchs (pretend to) think it will be.
The thing is that the kid living now that you’re ignoring in favour of zillions of future digital people in another galaxy millennia from now might just be the kid who helps makes that possible. Help her get a university education and she might be the inventor of the electro-magnotomic fitzis that ends up being a core part of interstellar travel. Or at the very least she won’t be one of the main figures in the revolution that forces people like Bostrom to work in the radium mines of Io.
Musk takes ‘crusade’ to Europe with Armie Hammer film
This assumption is doing a lot of heavy lifting… After all, if virtualisation / mind uploading is possible, then it’s probably possible to do it non-consensually, in which case the majority of those “10^38 digital people” might end up enduring a fate worse than death.
Anyway, we can trivially dispose of the idea that Nick Bostrom actually believes that we have a duty to maximise utility by maximising the number of people in existence by observing that he only has one child. What about the loss of all of those other children you could have had, Nick? Don’t they count for anything?
One of my favourite YouTubers recently posted a a somewhat relevant video essay: Those Transhuman Bastards”. (Re-titled for YT as “The world’s worst people have their own special ideology” because YT doesn’t like the work “Bastards”…)
This “never mind the people suffering now, think of all the countless future people who will totally exist if we let billionaires build rockets and data centres instead of paying taxes!” reminds me of something in a recent post by Cory Doctorow:
There’s a kind of Billionaires’ Syllogism at work here. That is, if the material conditions you enjoy are based upon the assumption that other people are not real; furthermore, that future people are doubly not real (by virtue of not existing yet); therefore, it pays to concentrate on the future people in order to maintain those conditions.
@6: I don’t agree that future people count for nothing, insofar as I think that we’re obliged to leave a livable world for our descendants. Where I differ with Bostrom &co is I don’t agree that we’re obliged to bring some arbitrarily large number of people into existence in the first place, just for the sake of maximizing some ill-defined “value”. People that exist ought to be happy; but there is no obligation to bring anyone into existence just in order that they can be happy.
Despite being generally consequentialist, I violently disagree with the simplistic utilitarianism that adds up everyone’s individual utilities to yield some aggregate number which we’re supposed to worship and serve. As Christine Korsgaard (a Kantian) puts it, well-being and ill-being must “tethered” to the individuals experiencing it; summing it up yields a meaningless abstraction. Thus, a society consisting of a quintillion happy beings is not a better state of affairs (on account of the mere number) than one consisting of only a billion. So: fuck utility-maximizing futurists.
Musk is said to be a fan of The Hitchhiker’s guide to the Galaxy The modesty of Zaphod Beeblebrox and the moral profundity of Ford Prefect perhaps?
@3: I am actually puzzled by Bostrom’s position, since he is also the author of the short essay “Pascal’s Mugging” which (unless I have badly misread it) is a rebuttal of this kind of simplistic application of expected value reasoning. A wildly improbable outcome (assuming we’ve even got a defensible estimate of that probability in the first place, which I strongly doubt) does not become the rational choice to pursue just because we make up even bigger number as the value of that outcome.
By their logic, them allowing a single child to die a preventable death today kills millions of potential future people, thereby making them horrible monsters.
Without following their logic, I agree with that conclusion.
@5: When EA points out that it’s umpteen times more cost-effective (as measured in meals-on-tables) to give $100 to the local food bank than to buy $100 of groceries and give them that, thanks that’s useful to know. When they ask whether I should give $100 to help one person locally or provide roughly the same level of help to 10 people in Bangladesh, okay that’s a genuine dilemma to ponder — can I show some favoritism to people who are socially close to me? Maybe, maybe not. But worrying about the fate of some literally astronomical number of hypothetical future people, most of whom are virtual persons existing inside computers (which we don’t know for sure is even possible)? That’s simplistic utilitarianism combined with a heavy dose of science fiction, on steroids. Not worth taking seriously.
Reginald @15: Why the fuck would anyone want to watch that POS when we already have the “Taken” movies?
… some 10^38 digital people …
If these entities even slightly resemble Homo sapiens, that means some very low (no notation necessary – possibly just 1) number of individuals and 10^38 subroutines, following some giant game of Core War based on competitive cyber skills with little correlation to physical surviving and thriving.
At least the remaining biomass scattered across the “colonized” planets may retain cellularity as it starts over, and its sentient progeny may learn a little from the wreckage.
Apologies if, as seems likely, this has already been dealt with and surpassed by the material found in birgerjohansson’s # 9 & Raging Bee’s # 13.
Meh. While biological ‘legacy’ humans will be around, real General Artificial Intelligence will be increasingly important. They will serve as a bulwark against the more harebrained ideas.
.
BTW the one-page short SF stories published at the vack of the Nature journal is more full of original ideas that the Muskosphere.
.
It is too bad we lost Freeman Dyson in 2020. He was one of the few futurologists that had a good prediction record. His 2000 The Sun, The Genome and the Internet wss right on the money.
@12: … and I’m sure only Roberts and his ilk truly know what those future people wish to do with their rights.
Got a sci-fantasy setting kind of Tron and Transistor inspired I’m writing for an RPG. Throwing out a lot of hard science to get to the point of “wow, neon people in cyberspace,” so please withhold musings about the implausibility of mind uploading: I know, not likely to happen. The hidden lore of the setting is that the people were uploaded for a space colonization effort and needed to have their minds kept active during the long trip. The problem is that the simulation and their minds have changed so much that the Administration secretly decided they’re no longer capable of interacting with the physical world. The ship’s been orbiting a star, collecting solar energy to continue the simulation and just let them live until the system inevitably breaks down.
Kind of a little jab against space colonization for me. Also an excuse for me to explore a “world in a small bottle” setting.
@9: “Tunnel Under the World”. Fredrick Pohl, Galaxy Magazine 1955. X-Minus1 episode in 1956.
10^38 is a 1 followed by 38 zeroes (not that that changes any of Torres’ findings or analyses.)
OK, good. Now we know for certain that Bostrom is a many-faceted arse.
Birger,
Close, but no.
cf. https://en.wikipedia.org/wiki/Simulated_consciousness_in_fiction
—
I notice the conceit of many future ‘personoids’ is much like the Basilisk of Roko, just a bit tweaked.
Right? The retrocausal leverage mechanism is common to both arguments; the bizarre ‘philosophical’ claim that the mere existence of hypothetical future simulants separated from us by millions of years is supposed to dictate our actual behaviour in the real world today. Quite ridiculous.
I also notice that future digital people are both non-existent, but the very compound term is oxymoronic.
And, if they are digital (which sounds primitive), then they can be copied noiselessly.
(Imagine 1E23 copies of John Morales roaming the datasphere!)
(Imagine 1E23 copies of John Morales roaming the datasphere!)
I don’t think that would trigger the techbros nearly as much as ONE copy of Greta Thunberg roaming the datasphere.
LOL.
I remember that story almost perfectly even though I haven’t read it in decades.
At the time it was a well known story and anthologized many times.
WTF do these morons keep appearing from?
As #14 observed, the most efficient way to ensure the well-being of the hypothetical future people is to do everything possible to ensure that everyone has the physical, intellectual, and social resources, needed to thrive. It is strange that these ‘effective altruism’ advocates seem unable to grasp this. Almost as if they don’t understand their own motivations.
Future altruism AFAICT always, always, involves present selfishness. There has to be some easily understood and devastating philosophical retort, some contradiction to be pointed at, to future altruism therefore. Right?
I mean in terms of something like Zeno’s paradox. You always have to cross another halfway point. Resolved thusly: The paradox is that while one is on holiday leave from a job, one is still accruing holiday leave. and so, if you work right till the end of your last day, well, you have to take leave, as much as you have at that time. But then you have accrued more at the end, so you have to apply again, and so on. The solution is to work backwards instead. Suppose by friday end of day, I would have exactly 3 days leave accrued. I finish work tuesday afternoon, wave goodbye and never turn up again, I don’t quite have a full 3 days at that time, but by the time friday arvo rolls around, the lines cross and it all works out.
Sorry if that seems obvious, but it had to be explained to me once and I am trying, laboriously, to make a point. Being, is there some similar dealing with (near) infinities in an easy to grasp way that deals with future vast amounts of humans vs what we have now that makes the present selfishness of future altruism look just as obviously and entirely silly as the friday arvo Zeno’s thing above does? Intuitively it seems like there oughta be, but trusting to common sense without backing it up is not a thing I like to do.
Sorry for double post, I should have thought it through more thoroughly before I posted the first time. I think I have it, or rather I can better articulate my question at least, now.
indianajones, that’s not bad.
Right there’s your pivot; your own words, as a proposition.
Basically, pose the converse: is it future altruism if it does not require present selfishness?
A discriminator. If someone says ‘yes’, then they are ignoring an universe of possibility.
Otherwise, they can’t sustain the claim.
There isn’t just one thing wrong with the idea of 10 exp38 digital people in the entire universe at some indefinite point in time.
There are many problems and they all add up.
A few obvious ones.
.1. We may never develop interstellar space flight to colonize the galaxy.
We know now that it will be very difficult.
.2. We may never develop digital posthumans.
.3 People might not want to live in simulated computer environments.
.4. We may find out the universe was already colonized by digital postaliens and they aren’t looking for neighbors.
.5. The whole idea that more people is better is an assumption and maybe it is wrong.
Today in most of the world, the birth rates are below replacement.
A whole lot of people don’t see any real reason to overpopulate the earth and can’t be bothered creating real humans.
.6. I’m sure anyone could find more unsolved problems with the whole idea here.
Given the number of unsolved problems and unknown hypotheticals with the idea, it is useless to use it to plan out anything in the here and now.
None of these problems are relevant to the 8.2 billion people living today. They are real people with real problems that we can solve now.
birgerjohansson @11
Fixing genes seems deeply problematic. Lets say we eliminate the genes for white skin now that vitamin D is environmentally available. Lessen the rate of skin cancer by a lot. What happens when civilisation falls and vitamin d becomes environmentally inaccessible lo subsistence farmers in high latitudes ? These people will have life long rickets .
Removing genetic variation removes the fuel of evolution.
Imagining that the human created genetic variations are not going to go down the line of the pug dog or thoroughbred horse in search of better is a big ask to me.
As to cryonics . Killing and then freezing a person , keeping them frozen for decades or centuries, then defrosting the person in a way to not cause any damage to the body, then restarting brain function is a chain of events that needs a bit of proof not hand waving.
.3 People might not want to live in simulated computer environments.
Which raises several additional questions; such as ‘Who is designing the simulated environments, and with what priorities?’ Not to mention ‘How much storage and multi-processing power would be required to host a sufficiently complex and realistic cyber-environment for even a thousand “post-human” entities, let alone billions?’ And how much material resources will be required to maintain and power such cyber-environment(s), PLUS whatever security measures/forces sufficient to keep the power going and the cyberspace platforms growing?
This simulated computer environment will NOT be a post-scarcity world. If the posthuman entities inside the simulation are to be kept alive, active and happy, then the meat, robot and cyborg creatures in the real world will have to keep on forever providing materials and labor to keep the lights turned on and safe. (Again, see “The Uploaded” by Ferrett Steinmetz — where the post-scarcity cyber-fantasy-paradise requires an ALL-scarcity meatspace population obediently slaving away in the hope of getting in on the cyber-immortality promised to those who prove themselves worthy.)
Raging Bee, from the turn of the century:
https://www.antipope.org/charlie/blog-static/fiction/accelerando/accelerando.html
I mean, it’s all nonsense used to excuse their terrible behaviour… but it’s also not even internally consistent. If you spend $1 trying to help some imaginary friend from a billion years in the future, then you spent $1 on 1 person. If you spend a $1 remotely improving someone’s survivability now, then by comparison from that billion-year standpoint, you save that person and all of their descendants for the next billion years. The trick is, they don’t want to spend $1 to help future people; they want to spend it on themselves, with the vague excuse that making their life better now will trickle down to future people. I think we’re all pretty aware of how trickle-down economics works at this point. And all of this working from the ludicrous precondition that maximizing population is even a good goal, muchtheless the greatest one. Lets wipe out humanity to make more room for bacteria! It maximizes future population!
@42 Trickle down economics but temporal. Proof by example, kinda like the book ‘A libertarian walks into a Bear’. Love it, ty @devnll.
@18,
That’s good, because that’s not what I said. I said their net present value to us is zero. It’s an accounting joke.
Astrolad@25,
The people in “Tunnel Under the World” are miniture but physical robots, not living in a purely digital simulation. In Simulacron-3 (also published as Counterfeit World), a 1964 science fiction novel by Daniel F. Galouye, does feature such a simulated reality (in fact, two layers of simulation).
Another of Bostrom’s silly ideas is that we “almost certainly” live in a simulation. I don’t know who he pinched his “argument” for that from.
KG: He most likely “pinched” that idea from his own inability (or knowing refusal) to understand that other people are as real as he is.
Related: https://www.vox.com/politics/496929/suicidal-empathy-heatmap-study-debunk
↓
It’s also a convenient new spin on an old idea. The billionaire tech right cohort Musk belongs to has long looked for a way to unshackle itself from the leviathan of basic social responsibility. With the concept of suicidal empathy, they have found an intellectual framework to argue that their policy preferences and bottom lines aren’t just about greed or self-interest, but a matter of cultural survival.
In their quest to make this argument, many on the right have pointed to a study that they see as a sort of intellectual checkmate. In 2019, a group of psychologists gathered participants across political ideologies and asked them to express their level of concern about different groups ranging from their close family to animals to all things in existence. In one section, you see a series of heatmaps that appear to show that liberals focus their moral concerns on groups further away from themselves, whereas conservatives value their close loved ones.
[…]
Kyle Fiore Law, an assistant psychology professor at the University of Utah and one of the authors of the study, decided to embark on the research in part after seeing the heatmap memes.
“The assertion going around was that liberals had come to care more about strangers and distant causes than about their own families and communities, and that claim is testable,” he told me. “It also sat awkwardly against decades of evidence that favoring close others is a widespread default in human social cognition, so we wanted to check it directly rather than assume either side of the debate was right.”
The problem, in Law’s view, was that the 2019 study was not robust enough to really tell us who liberals and conservatives prioritize.
“The 2019 studies measured how far moral concern extends, and they did that well,” Law wrote to me in an email. “They were not designed to detect whether concern for distant others competes with concern for close others, because both tasks built an outward-expanding structure (e.g., from close humans to distant humans to non-human life to eventually the natural world and non-sentient targets) into the measure itself.”
Raging Bee@46,
Not the idea, the argument. He does have one, but it’s fatally flawed. Briefly:
(1) Technological progress can be expected to produce world-simulations including huge numbers of digital beings to exist in the not-too-distant future.
(2) If this happens, the vast majority of thinking beings will be denizens of such simulations.
(3) We have no reliable way to tell whether we are in a simulation or not.
(4) So either such simulations will not come into existence or, simply on the grounds of probability, we are probably living in one.
The flaw: (1) depends on the assumption that we are not living in a simulation. Unless we make this assumption, we have no reason at all to think (1) is true. If we are in a simulation, the laws of physics could suddenly change, time could bend into a circle, the moon could hatch into a giant space-turtle, it could end tomorrow when God’s mum calls him down for tea. Moreover, we have absolutely no way of assigning probabilities to these and infinitely many other possibilities.