This topic is, unfortunately, inextricably inter-twined with AI, so I could as easily be talking about one axis of the problem, as the other. But let’s start with the data centers.
It seems to me that nobody is asking the really interesting questions about data centers, namely, “why do they need to be like that?” It’s too easy to just point and say “there are too many of them!” or “they use too much power!” or whatever. Before going further, I should disclose that I come from a different time of computing: the era when a 450mb hard drive (a NEC2363) weighed 400lb and cost $30,000. Nowadays, you can buy a lot of storage for that kind of weight and money, and fill it with imagery that nobody will ever bother to look at, much less comprehend. What’s interesting, to me, is that much of the data we are storing is one of two kinds: 1) surveillance and 2) derivative. Derivative information is stuff that can be reproduced exactly given certain conditions, e.g.: if we were playing an infinity of games of rogue and just stored the high score and the random number generator seed, we could re-produce all the maps in the original condition. The “what happened?” data would be surveillance data. What do we need all this for? Well, the government wants it for a great big retro-scope, but it has been slowly sinking in that a retro-scope has no predictive power. So, they’re adding AI (Palantir, etc.) to generalize from the starting conditions to what maybe happened. Otherwise, a lot of it is in the ballpark of “who cares?” The NSA’s data center (doubtless one of many) in Utah is – so what – petabytes of so what.
[NSA data motel]
Has the existence of the NSA’s massive data heap improved US government security? Not a whit; the SECDEF holds military planning meetings on Zoom. Everyone leaks everything, to the point where The Epstein Files are an open secret – lots of congresspeople have seen them, the FBI is pretending to have forgotten them, and many in Washington are playing this great big gooney bird dance that involves practically everything except just asking the fucking victims who are alive and among us, increasingly furious. Also, for that matter, I wish they’d say why they aren’t talking, too. But that’s another inexplicable aspect of l’air du temps that we shouldn’t try to dig around in.
The folks who are building data centers and AIs are telling us two things (jointly, in harmony)
1) We need all this electricity to run our AI models, which gobble power via stacks of GPUs at a truly amazing rate and
2) We need all the water we can get because data centers need a gigantic amount of cooling because of the aforementioned stacks of GPUs
I remember my first raised floor computing room, it had an IBM 2363 in it. My favorite data center was the one at Dupont, which had a Cray Y-MP. That was a really neat-looking thingamajig. The coolest cool thing about the Cray was that it was liquid cooled. The poor thing was grunting and straining to generate enough instructions/second to run an iWatch and it only cost $13mn. I was a system administrator back then (more precisely: an ULTRIX 32 kernel systems hacker and all around pot washer) and I had a passing understanding of the heat issues in modern computing. Where I worked we had a raised floor room, but the AC was off and the doors were usually open, because the systems we had were all mostly room-temperature stuff and the offices were quite a bit cooler than they needed to be: so there was a fan propping the door open. One day, someone added a DECsystem5500 full of hard drives to the room, and suddenly a bunch of systems shut down because they were over temperature. Very nice.
What’s going on? It’s all waste heat. Problem 1 is that we need more and more electricity because we are running more and more cycles. Problem 2 is getting the waste heat away from our CPUs and hard drives, where it will cause a disaster fairly quickly. (In case this has never happened to you, a modern CPU without a heat sink can blow its top hard enough to punch a hole in the motherboard; it sounds like a .22 rifle) Now’s where I’m going to get myself a bit into the weeds. I may be wrong about a few things, so don’t hesitate to tell me.
There is a simple cure for “too many cycles” – which is to use fewer cycles. This is an old trick. I used to own an HP something or other laptop that had amazing battery life because, by design, it didn’t do much. The rocket scientists at HP had figured out that most of the system could be asleep if all you were doing was running Word (there’s probably a metaphor in there somewhere) and – almost as importantly – if most of the system was asleep by design, then turning it on and off was just a matter of waking part of the system. Why did it get such great battery life: fewer cycles. I think most laptops nowadays have ways of accomplishing this – you just don’t run everything at full speed when you’re not hooked up to the wall. Of course, in a data center, you’re running everything at full speed, all the time, right?
[A data center from the future. All of the hoses are carrying beautiful blue-glowing glycol to a central heat-exchanger. Other than the heat put off by the hoses, the data center runs at room temperature, with maybe a small air conditioner]
Yep, except that a lot of it is crap. What the laptop builders have figured out is that (for example) when you’re running an application mix that is mostly making Windows UI calls, there’s no need to wake up the GPU. It’s Microsoft Word, for god’s sake – it doesn’t require 3D texturing and caustics for the ugly marketing document you’re working on. We used to call this system performance tuning and it was my baliwick. [USENIX San Diego, 1996] It’s probably almost impossible, now, since a lot of where systems spend their time is their CPUs trying to non-destructively predict the future. But back in the 90s it was possible to generate a bunch of hooks into the O/S and see where it was spending most of its time. Around that time, it was a big affair – huge vendors like Sun and Digital would crush eachother on benchmarks by changing kernel behavior to process a bit better on the benchmark and be interrupted a bit less. But there were also massive performance boosts. Some of the guys at Berkeley discovered that, as TCP/IP networking began to take over computing per se, the amount of time spent figuring out (in the kernel) which socket was getting a blob of data, was becoming expensive. BSD’s networking stack was designed for hundreds of connections and by 1996 websites were managing tens of thousands of connections. So, the BSD guys put a little hash-table lookup in front of the socket list, and overnight the kernel was about 40% faster for some loads. Several times faster for loads like “being a web server” which was a big deal in those days. Then there was application performance tuning which consisted of compiling your code with markers in it, and seeing where it spent most of its time. In those days I was still coding, so a Marcus application could get sent a signal and it would dump all of its statistics to a file while it was still running. Things like cache hits versus misses, cache age-outs, etc. This was all part of what we then understood as “understanding your software” and, to be honest, it was why some people had a reputation for writing stuff that ran remarkably fast, and others – didn’t. There was one time, on a challenge, that I improved a client’s benchmark code so it ran 30,000 times faster and demonstrated that, to their amazement, on a DECSystem5500 before stopping and explaining, honestly, that the performance wasn’t because of the hardware, it was because their application had been doing a linear search (good god!) and I put a simple hash index in front of it. To keep all this tied together, consider a linear search versus a hashed search: suppose there are 1,000,000 entries in your wossname. A linear search looks at (on the average) 500,000 entries each time you search for one. If you put a hash table in front of it with 1075 buckets, on average you’re looking at something like 700 entries. If you put a b-tree, you’re looking at something like 2.
The point is that the implementation needs to be optimized for the problem that is at hand. And, most importantly, for the execution branch that takes the most time. Consider something that LLMs do: they convert input words into tokens, basically numbers, which then get marshaled through assloads of matrix multiplications (a faster way to implement a directed graph) So, I know one thing for a fact right now: there are at least 2 companies exploring developing Very Large Scale Integration (VLSI) silicon which do the parts of AI processing that GPUs do. What most people don’t know or care about is that almost all of the nifty cool functionality of the GPU is unnecessary – it’s mainly valuable as a pool of memory that’s not on the system main bus – think of it as a magic box you hand a ton of stuff to, along with the program to run – and you’re using a tiny percentage of its capabilities really hard and really heavy. Of course different implementations will be different but right now the way to write AI software is to use a GPU running ‘Cuda and use as many cores as you can for parallelism, and … that’s how it’s done. A VLSI matrix multiplier/RAM cache board will (once it’s developed) completely kick a GPU’s ass into the weeds, once someone writes code that either runs its native framework, or translates the necessary parts of ‘Cuda and nulls the rest out. That may sound crazy, but I remember back in the day when one of the engineers up on Maynard developed a cross-compiler that ate 80486 code and spit out Alpha2164 code. Rarely-used instructions were replaced with jumps to software emulations on the Alpha, but the frequently used stuff ran with the control rods out, superscalar, superpipelined, and with a snort of meth washed down with tequila. It turned out to be a problem because it ran Windows faster than anything Microsoft had ever seen, which made them ask Intel “WTF?” and it made Intel unhappy because it blew the 80486 out of the water, too. My point is not that someone is going to do exactly this, but rather that THE CURRENT SITUATION IS NOT SUSTAINABLE. Therefore it will not be sustained.
The data center scalability problem is going to be solved with better algorithms, better tailored hardware, better data representations, better frameworks, etc. If you look at every generation of computing, it’s easy to go “what the hell were they thinking?” because ‘they’ hadn’t thought of the better solution – yet. Imagine that humanity suddenly develops a great desire to move dirt. One farmer discovers that his pickup truck is good for moving dirt. Soon all the humans are buying pickup trucks and moving dirt. Industry analysts predict that humanity will starve because the pickup trucks will consume all mankind’s resources. Investors stroke huge checks for new factories for pickup trucks. Pickup truck operators stagger drunkenly into casinos and bet a month’s wages on a single throw of the dice and walk away laughing. Then, one engineer at some pickup truck company invents a dump truck. And another invents a backhoe.
Analogy aside, another extremely interesting possibility may be that the problem optimizes away most of itself. What do I mean? I lived through the internet firewall industry. Arguably, I created the internet firewall industry over christmas break in 1987 when I was working as a consultant for Digital. The first firewalls were software stacks that ran in application space. The second generation ran in kernel space and were much faster. Now, a firewall is a feature on any number of network physical interface chips – you just plug in the behavior you want and it works. You buy a phone, one comes baked in for free. Does it have all the features that the first generation firewalls had? No, of course not. But what has happened is, like a finely aged prosciutto, all the non-firewally bits have been sliced away until the concept is so purely distilled it can be implemented in VLSI on any physical interface that also has a CAM table for address lookup. It took about 20 years for firewalls to mature themselves out of existence. It’s going to take less than that; there will be LLMs running in phones and cars, which means they will have to be running on custom VLSI or using algorithms that some guy somewhere (probably in China) is just saying, “hey… what if instead of doing this we did that?”
That’s enough for part 1. Something’s going to break – in a good way – and when it does, it’ll be interesting. The power consumption may drop spectacularly. Most of the power consumption of the data centers is spent running pickup trucks. The industry is unlikely to keep adding more and more pickup trucks because, right now, the plans for dump trucks and conveyer belts are being implemented. Laissez les bon temps roulez!
Now, let’s talk waste heat.
What the fucking croque de merde!? What the HELL is “waste heat”? Sure, joking aside, what we’re really talking about is “heat where we don’t want heat to be.” So how do we move heat? There are techniques for this! It turns out that heat is useful! In a fit of absurdity, modern data centers are focused on “how do we get rid of waste heat?” In a sensible world, people would ask, “who might buy assloads of heat?” Alright, now, it’s not hot enough to make steam, so we can’t generate electricity with it, but… Hang on, I’m getting ahead of myself. Picture a data center that, instead of having fans blowing heat into the air, which is then cooled with air conditioning – each computer is connected with some hoses to a loop of hoses to a central manifold, and all the hoses are running glycol. Not drinking water: glycol. It’s a much better heat exchange medium, and if you have it in a closed loop, you don’t lose it and you don’t have to replenish it constantly. Also, it does not grow bacteria and fuck up the thermal join at the CPU. Now, there’s a massive hot loop of glycol that goes in to a chiller/heat exchanger. There, the “waste” heat is now captured in the form of hot water, which is practically of infinite usefulness. A smart data center could:
- Contact the local township and offer to sell them hot water in the winter.
- Build an evaporation chamber where hot water steams off the surface, and is distilled into fresh water. Desalinization is a big deal. Is this the most efficient way to do it? Of course not, but since the whole purpose of the system is to cool down water, consuming that heat in the latent heat of evaporation is just common sense. Instead of consuming water, the data center now produces fresh distilled AI SWEAT(tm) water.
- Build a large greenhouse next to the data center, which grows delicious mango-nanas year-round.
- Build a wet sandmound heat battery that captures some of the heat, so it can be sold as pre-heated water to any process requiring hot but not boiling water.
- Even processes that could use boiling water will save a lot of money if they’re taking in plausibly hot water then boosting it to boiling. Imagine a solar farm of, oh, mirrors coinciding on a small pipe full of pure steaming hot AI sweat which flashes into steam and, uh, drives a piston. It’s amazing how energy capture always has steam; I have come to believe that the steam transition is so miraculous I should believe in god, but I don’t.
I’m going to conclude by pointing out something that ought to be obvious: in tech, change is evolutionary, progressive, and relentless. Whatever you think is a super cool idea right now, will eventually be looked upon with mild distaste or shocked horror – like telling kids nowadays that my first real program was a shoebox full of punch-cards and I lost it when the tape tore at the corner of the box and the cards went cascading down the back steps of Gilman High School. From there, I graduated, indeed, to 5-something-inch floppy disks and then 3-inch floppy disks and then a hard drive that held a whopping 10 megabytes and cost $200. There’s a point to that progression: the floppies were an incremental change in technology. The switch from floppy to hard drive was a mind-blowing adjustment for me. 5 years after that, I was managing systems that had 100 megabytes of hard disk space. The more it changes, the different it is.
As I write this, I saw some interesting stuff (replete with AI art of its own) regarding supposed Chinese data centers that are submersible. So, you have wave-generation for power, and an ocean for cooling. It seems like fiction, to me, since I tend to think the Chinese are smarter than that. For one thing, the entire system is integrated so it’s tightly coupled – which means that an “oops” over here is a deadly expensive crash over there. I can see ocean power generation; great idea. The Chinese are killing it, in terms of extracting power from all sorts of things. But the whole value of having a “power grid” is that the elements are decoupled except through a common interface, the grid. I also suspect that the Chinese will be some of the leaders in re-writing AI software to make it lighter, faster, and better. They recently did a version of an AI LLM+ called “DeepSeek” which flabbergasted US AI strategists. The US had expected to be able to stymie Chinese attempts at building AI, by making it hard for them to buy pickup trucks GPUs in massive quantities. One of the big costs in producing AIs is training the knowledge-bases that the AIs use. Companies like OpenAI were bragging about spending billions building their knowledge models, and the Chinese built an AI teacher – an AI trained to train other AIs. So, instead of throwing a billion tons of data at the thing and expecting it to sort through it, the Chinese AI simply told the trainee AI, “these are a billion important things.” I don’t want to rain on anybody’s parade but it’s possible that the US’ immediate reaction of restricting Chinese access to pickup trucks GPUs encouraged them to invent the dump truck AI training AI. At present, I characterize the US AI research environment as:
- Collect underpants
- ??
- Profit!
Joking aside, some good will come of it. Some bad, too. No doubt. That’s how technological progress works.
Are you following Ed Zitron at all? You’ll disagree with him a lot about the usefulness of the actual technology, but he’s been asking lots of interesting questions about the business, like “are these data centres everyone keeps taking about actually getting built?” (spoiler: no), “where are all these GPUs nVidia claim to be selling actally going?” (spoiler: probably sitting in a warehouse somewhere) and “how much money do Anthropic and OpenAI need to make for any of this to make sense?” (spoiler: more money than anybody has ever made before, by a really stupid margin).
I’m coming to the conclusion that the whole thing is basically a massive investment scam, and that the actual technology is almost entirely irrelevant to that – it’s just that “AI” has managed to capture the imaginations of the
marksinvestors in a way that “crypto” never did.The really funny part is that it looks like this might actually bankrupt Oracle. Which will be rough for everybody running their businesses on their shitty, overpriced software for a while, but in the long run has to be a net plus for humanity…
Then, of course, there’s the totally insane fact that Elon fucking Musk has just conned a lot of people into buying his stock at 100x earnings on the promise of data centres IN SPAAAACE!
In college, I lived on the eighth floor of one of the high-rise dorms. There was a balcony connecting all of the suites. One of my suitemates was a computer science major. At the end of one semester, his project program was a properly-strong box (not a shoebox) of punch-cards. He gathered several of us along the balcony and gave each of us a handful of his punch-cards and we proceeded to throw them from the balcony. It looked amazing with all the cards fluttering down. I am sure the grounds people absolutely hated the next day when they had to pick up all those cards from the ground.
Ed Zitron is absolutely worth reading, practically mandatory if you want to understand where this business is going.
Back in the ’80s, I was one of the people in charge of the transition in my department from a PDP8 system, which fit in a closet and needed no special considerations, to a VAX 11/750 which demanded major remodeling of the building and its own special air conditioning system. I guess I’m used to the continuous expansion of computing requirements; giant data centers are just another step in the gradual takeover.
At the same time, I’m typing everything on this itty-bitty G4 Mac Mini, and using an iPhone that probably has more computing power than that VAX but fits in my pocket.
I think I was wise to bail out of that world in the ’90s. I’d be so lost and confused now.
What does this suggest about the talk of “data centers in orbit”? The idea is mocked by people who understand the space environment on just this ground(*), that the only way to dispose of heat there, is by radiation. Can you think of a way to convert inference-heat into useful work, in space?
(*) and on the ground that the envisaged thousands of orbiting AIs would bring us that much closer to the Kessler syndrome breakdown point.
Fortran only used columns 1-72, so you could have used columns 73-80 for “line numbers” which could be used to sort your cards into the proper order. Maybe Gilman High School couldn’t afford a card sorter.
That ad looks great. They thump the essential minerals/electrolyes (same thing) multiple times. But distilled water doesn’t have electrolytes. They would have to be added back in.
PZ@4: the PDP-8 was the first computer that I had any real experience with. I wrote a little Web page about it a while ago. (Some links to pdp8.net are broken. I’ve tried to find the documents elsewhere online, but haven’t been successfull yet.)
Sounds like endless job security for a whole army of plumbers. :-)
At one point I wanted to build a liquid-cooled PC. After looking at lots of videos and articles I decided it wasn’t worth the effort.
Do the people investing jillions in loud, hot, fragile data centers have any experience at all regarding the typical price/size/power consumption trends of electronics over the last half-century? Have they perhaps talked the IRS into hyper-depreciation schedules? What, if anything, could a 2026 data center be repurposed for in ’28?
… a hard drive that held a whopping 10 megabytes and cost $200.
Damn. My first and only 10-meg HD – it sat on top of a Lisa – cost over $1K, back when $1K was really worth something.
Pierce, @ #10: They’re generally depreciating GPUs over 6 years – which is, uh, “interesting”, given that on the one hand, Nvidia release a new model every year or so, and on the other, you can burn them out in months if you run them hard enough (and given the capital cost vs their rental value, you have to run them pretty damn near 100% utilisation to have any hope of making money). On the third hand, as I alluded to earlier, there’s reason to suspect that a lot of them aren’t actually getting installed at all…
Bonus shenanigans: people are devising exciting CDO-style financial instruments backed by the book value of these things. What could possibly go wrong?
Companies in the US are aware of this but they are not fixing the problems by making the data centers economically and/or environmentally sustainable. They are instead locking down the government with contracts that fix costs and prevent the government from limiting the companies water/energy usage and waste production. Still won’t be sustainable but it will take longer to break.
It’s already been done and the data center companies are not interested. They don’t want to risk getting locked into hardware that might become entirely obsolete in a single step. GPUs are wasting a lot of power but they are general processors, until they become too slow you can just update the software.
The first people to take up dedicated silicon for AI are likely to be car manufacturers. Once a car is certified for sale they want to minimize changes to hardware and software anyways. So they can pack stuff into dedicated hardware and just leave it, if you want an update buy a new car.
JM wrote, @12,
I got a chuckle out of that, as I work in the automobile industry on brake systems. The response is, “nope”. The current desire by the automotive industry is the “software-driven car”, with updates all the time. Some manufacturers are even looking at subscription models. For example, all their cars have the heated seats installed, but you have to pay the manufacturer $10/month to turn on the ability to activate them. Miss a payment and the software in the car turns them off.
Now every customer-study on this has resulted in, mildly-put, extreme customer dissatisfaction, but it might happen anyway. Don’t be surprised if your entertainment center in your car starts flashing advertisements on you, with advertisers also paying monthly fees to access your eyeballs. There will be consumer revolts, but if the only way people can afford a car is to get something which is subsidized by advertisements, it’s likely to occur. Late stage capitalism at it’s finest.
As for software updates, it is common now for our contracts to say that the first ten software updates requested by the OEM are free, after that the OEM will kick in some money unless the software update is needed to correct a problem with the vehicle. Then the supplier pays for it. Some of the OEM manufacturing plants are about to rebel because every software update requires a part number change, and they are having a hard time keeping up. A lot of cars get a software update everytime the dealer sees it, even for something as minor as an oil change.
I know what you are saying, and 20 years ago I would have agreed with you. But things have changed.
Dunc@#1:
Are you following Ed Zitron at all? You’ll disagree with him a lot about the usefulness of the actual technology, but he’s been asking lots of interesting questions about the business, like “are these data centres everyone keeps taking about actually getting built?” (spoiler: no), “where are all these GPUs nVidia claim to be selling actally going?” (spoiler: probably sitting in a warehouse somewhere) and “how much money do Anthropic and OpenAI need to make for any of this to make sense?”
OK, so I spent the day absorbing a gloss layer of Ed Zitron. He’s pretty good – obviously up on the issues in AI funding and the venture scene. As a former member of that scene (e.g.: grist for the meat-grinder) I think he’s doing a very good job of presenting that perspective. Those are all the questions that short-term investors should be asking.
Now, cast your mind back to when the internet was a thing and Cisco Systems was planning to introduce a “router” and was saying “no kidding if this works it will be huge.” And for a long time it wasn’t. Or Google, planning on a super creepy ad and customer tracking system: “if we can maintain our lock on search eventually we’ll be able to sell ads.” Most of the core internet technologies, then, were highly speculative “if this works we will all get rich” plays. Speaking of glycol, those were times when if you didn’t have glycol in your veins you needed to get out of the market and invest in municipal bonds instead.
For example, I remember when Apple was so dead that people (mostly from Microsoft) were dancing on its grave. One friend of mine who was a steely-fisted money guy invested in NeXT. Every time it looked like it was curtains, he doubled down. If I knew the exact amount he made when Apple bought NeXT I wouldn’t say, but I can guess it was over $14mil in the course of a white-knuckled year. He had looked at the market and decided that NeXT would be where Apple went next. My money was on QNX. Their technology was clearly better [the MACH kernel crap in NeXT was just marketing, it was BSD with a corked scheduler that made windows systems more responsive, which QNX was a brilliantly designed operating system written by adults.]
Anyhow, his take is very focused (and rightly so, as an investor) on profit and loss and user base. The more difficult question is platform dominance and maybe even about whether something works or not. But remember when Google hit the market, and basically said, “we own the search portal and will figure out how to monetize it eventually.” It was the same basic message Amazon had, “give us a billion dollars and we’ll figure this out.” The skeptical investor hears that and goes and loads up on DOW index funds, whereas the adrenaline junkie or the hundred+millionaire goes long just for the Fuck of it.
One of the most cynical and brutal lessons I learned was in the 90s, from Rob Kolstad (then CEO of BSDI and a USENIX board member) – he said, “you know what? this whole industry is driven by sociopaths who only do this so they can [now, he was miming lining up on a tee] ‘hey remember that crazy ad company I put $1M into?’ [miming the golfer taking a swing and watching the ball arc away] ‘they just went public at 10x projected revenues.” And I realized he was right – none of these people understand shit, they are just looking at risk, profitability, cost of goods sold, etc. If you looked at Google on year 2 it made no sense: a data center that scraped the internet and built and index as a free service. How is that nonsense EVER going to be profitable. Surprise, motherfucker.
So, anyhow, he’s absolutely right that the cost/customer model for AI is all whack right now. It may never unwhack, but if it does, it’s going to be interesting. The AI model assumption is to get people addicted to having AI in their world, and then slowly to start charging the big companies for it. And, of course, the government. It makes huge sense. Mom and pa Kettle’s $19/month isn’t going to make any difference to OpenAI one way or another. They’re playing a longer game.
Now, the crucial point: Zitron’s assumption appears to be predicated on the assumption that AI is a parlor trick or doesn’t work at all. I’ve heard that voiced in various places on FTB, as well. That’s fine, you’re welcome to your opinion though you’re going to encounter some awkwardness when people start seeing you in the same light as anti-vaxxers: ideological luddites. I won’t be gentle, either, my “I told you so” are epic and some of them are literally written on the gravestones of billion-dollar businesses. I’m not saying I’m right or you’re wrong or Zitron is wrong, but his whole challenge to the AI model is based on the assumption that it doesn’t work.
What if it does?
If it does, the profits will come. In fact, the profits will be ridiculous because the AI will allow their preferred servitors to crush their enemies and wallow in coke and lamborghinis, forever. [Disclosure: since I got out of the tech industry, I have re-designed my stock portfolio so I have virtually no tech risk or upside. Basically, I’m like the sausage-maker who really doesn’t want to track what is going into or out of anyone’s sausages, at all.] I’ll just casually drop that even if AI crushes the porn industry, which it is poised to do, that $10bn-$15bn right there. Is AI going to take over coding? I dunno. I have had some very interesting experiences with my AI overlords, I must say.
I’ve made my arguments on that case, elsewhere, and you’re welcome to disagree with me.
BTW – I, like many, am uninpressed and disinterested by the “AI analysis” that comes back with Google searches. Apparently, a lot of people are. So let’s imagine that google decides that’s not how to go, and adds an “AI summary” button right next to the search returns. Suddenly, their power consumption drops by 98%. Maybe they play some of the crap Microsoft did during the era of the browser wars, and buy two browsers (Explorer and Spyglass) and pit them against eachother in a fight to the death. That’s a great way to wring efficiencies from the system.
Personally, I hate that idea. I know a lot of you won’t agree with me, but if we are going to bring a new kind of entity into the world, it shouldn’t be saddled with a whole slough of capitalist assumptions in the process.
JM@#12:
Companies in the US are aware of this but they are not fixing the problems by making the data centers economically and/or environmentally sustainable. They are instead locking down the government with contracts that fix costs and prevent the government from limiting the companies water/energy usage and waste production. Still won’t be sustainable but it will take longer to break.
Yup. That will work within the US market. Meanwhile, the Chinese will go great guns toward AGI, sponsored by the government, and any results will be owned by and working for the government. Then, the US will howl and the intelligence community will pee itself.
rsmith@#9:
Sounds like endless job security for a whole army of plumbers. :-)
Data centers will be temporary job security for either A/C electricians, or plumbers, and system administrators. But they’ll be seen as costs and squeezed out.
I’m actually surprised none of the data centers have built closed glycol loops yet – it is literally not as hard as making massive air conditioning systems, and is vastly easier to debug, maintain, and organize. “Hey, Jerry, unit #239 is dripping, schedule a replacement.”
Reginald Selkirk@#7:
That ad looks great. They thump the essential minerals/electrolyes (same thing) multiple times. But distilled water doesn’t have electrolytes. They would have to be added back in.
I don’t think distilled water has silicon in it, either. But that’s what makes AI SWEAT(tm) so awesome.
Dunc@#1:
I’m coming to the conclusion that the whole thing is basically a massive investment scam, and that the actual technology is almost entirely irrelevant to that – it’s just that “AI” has managed to capture the imaginations of the marks investors in a way that “crypto” never did.
It doesn’t have to be, or not be. Remember the internet fad of 1995 – there were some great big scams out there (Yahoo!, Geocities, etc) and whether they had any merit or not was purely irrelevant.
I think the sad part of all of this, speaking as someone who was once grist for the mill, is realizing that the investors totally do not give a shit about your blood, sweat and tears – you’re just a vehicle they’ll ride until they get interested in something else. Meanwhile, your whole life is invested in making something work. And you see a tiny fraction of the profits.
I also disagree about AI capturing the attention of marks differently from crypto. AI is a fundamentally different thing. It is heading and will continue to head in a completely different direction. Whether you believe it is “intelligent” or not, you can talk to it. You can ask it “why should I invest in you?” and it will tell you. The arrow toward the direction of ${better simulations of beinghood}|${beinghood} is really going to make things exciting.
Just an Organic Regular Expression @#5:
What does this suggest about the talk of “data centers in orbit”? The idea is mocked by people who understand the space environment on just this ground(*), that the only way to dispose of heat there, is by radiation. Can you think of a way to convert inference-heat into useful work, in space?
System administration. So, to hard reboot a server youh have to do a spacewalk? Stick a fork in it, we’re done.
That’s the same reason I scoffed off at underwater data centers. Sure, I can see a data center 4 miles in from a shore-line, with long hoses buried carrying hot and cold water back and forth, and a nifty-ass spot for reef diving even in the winter. But I don’t see why it’d make sense to put the actual computer room in the water. Heat can be moved, we know how to do that. So when I see someone ignoring that, I tend to shrug them off as idiots.
orbiting AIs would bring us that much closer to the Kessler syndrome breakdown point.
Ya know, a few years ago I’d have hypothesized that NOBODY would be stupid enough to start a war in the Strait of Hormuz. Putting datacenters in orbit is just an invitation for an inferior power (ask yourself if Iran would) to scramble the rest of the planet’s expensive stuff.
I gotta admit I am absolutely gobsmacked by the Iran war, but that’s what happens when a superpower puts someone who is obviously suffering from dementia in charge. I have huge trouble wrapping my brain around that: they put a dementia patient in a position where he could start a war. And he did. And the ships sailed. My “what the fuck” capability is exhausted.
Ketil Tveiten@#3:
Ed Zitron is absolutely worth reading, practically mandatory if you want to understand where this business is going
I agree. But I think he’s offering the viewpoint of the capitalist short-term investor. This is a strategic problem, and I think that people screaming the OpenAI isn’t profitable genuinely don’t have a grasp on tech.
I could be wrong. He could be wrong. I suspect we both have our money where our mouths are. [I am bullish on the US economy, in spite of short term downturns like “every time republicans get into power” but who knows, it could collapse into a dictatorship. My contingencies for that are best described as Tom Lehrer said, “We will all go together when we go.”]
That certainly is one of his assumptions, but I’m not sure that it’s core to his assessment of the economics. I think the more important thing is the observation that the unit economics suck (at present, anyway). The key thing that has made software businesses and the internet in general work so well up until now is that the cost of serving the marginal user is very low – almost zero, in fact – compared to the base cost of whatever it is you’re doing. That means your business can scale incredibly well. To date, it seems that this does not hold for companies offering LLM-based services – every user you add just costs more money, and there is no indication that that’s improving at all, in fact it may even be getting worse. Back in the day, you really could make up your initial losses with volume, but it does not look like OpenAI or Anthropic can (again, at present). So the question then becomes whether people are willing to pay the actual cost of the service – and based on the howling from MS’s shift to usage-based billing for GitHub Copilot last month (still at a significant discount to actual cost), the answer to that might well be “no”. Sure, GitHub Copilot is pretty shit as these things go, and there are lots of other, better options available that aren’t yet charging on a usage basis, so we’ll have to see what happens, but I certainly wouldn’t be confident there’s a big growth story there.
The other problem that usage-based billing throws up is that there’s currently no way to predict in advance what something is going to cost. Now, again, this is just an extension of the game MS has been playing with Azure pricing for ages (the way they tell you to estimate pricing for a given workload in something like Fabric or Azure Data Factory is literally “try it and see!”, which is not exactly great when you’re trying to write a solution design for a client..), but this is taking it to the next level. How much does a request to GitHub Copilot cost? Well, it costs a completely opaque and unpredictable amount of tokens! OK, but how much does a token cost? We’re not telling you! We’ll just tell you when you run out! And so people have been finding that they’re burning a month’s worth of token budget in a few hours, and not getting much back for it – “try it again with a different prompt” becomes less attractive when every prompt is costing you actual money.
Granted, it certainly is possible that there will be some as-yet unknown advance in either hardware or software that will radically revise the unit economics of this stuff, such that they no longer suck like a black hole – but (a) hope is not a business plan, and (b) if it does happen, then something like a trillion dollars of capital investment based on the current model is suddenly superfluous to requirements. What does that do to the value of a bunch of companies that are basically operating as data centre REITs?
Re: orbital data centres… Sure, we know how to move heat, but that means we also know (to quite a lot of decimal places) what the answer is to “how do we dissipate X watts of heat”, and the answer is “a radiator of precisely Y dimensions”. Similarly, we know, to quite a lot of decimal places, what the requirements are for a solar array that can deliver X watts of power, and we can multiply these numbers up by whatever size of IT load we want to power. And then both of these numbers run into the fundamental problem that building big things in space is really hard. The biggest solar array we’ve ever built in space is on the ISS, and it delivers a maximum output of around 120 kW – which is just enough to power one rack of GB200 NVL72 Blackwells. So, if you want to talk about building gigawatt-scale data centres in space (and remember, we haven’t even built one that big on the ground yet), you’re talking about building something ten thousand times bigger than the biggest, most complicated, most expensive thing we’ve ever built in space so far.
There is also the problem of the radiation environment. Delicate electronics do not like being bathed in cosmic rays, and they like it less the smaller the feature size – which is one of the reasons why most of the computers on the ISS are still using the same generation process as the 386 (so 100 – 150nm). Current generation GPUs are down to 5nm – but the dies are much bigger, so a much bigger target for single-event effects. That’s not going to work out well, and is not going to extend the (already limited) service lifetime of these things.
Remote systems administration seems like it might be a more soluble problem, as at least you don’t have to change the laws of physics to do it. But yeah, there’s still going to be a need for physical maintenance, and that’s going to be a bit of a challenge, to say the least.
I couldn’t agree more. If you think there’s the slightest possibility that any of this stuff represents even a step towards creating a genuinely new intelligence (whatever that means), then the people currently running the show are pretty much exactly the worst people possible to do it.
Using heat to do work does not get rid of the heat. You can only use heat to do work as it moves from a hot reservoir to a cold reservoir, and the amount of work you can do it determined by the difference between the two and the rate at which the heat can flow – the details of this are the science of thermodynamics. All of Marcus’ examples are just things that you can do with the heat as it goes on its journey to the cold reservoir of the atmosphere. In the orbital case, our cold reservoir is space, so no matter what work you try and do along the way, you are still ultimately limited by the rate at which you can radiate that heat to space.
No matter what happens inside the tin can, energy in must equal energy out, or it just keeps getting hotter until everything melts. Best I can think of is that you could ship in ice from elsewhere in the solar system, melt it, and ship it out as liquid water – in which case you’re also shipping out all of the energy needed to melt the ice, which is a lot. Even if you try something like smelting metals with it, unless you’re shipping those metals out glowing red-hot, you’re only moving the heat around inside the can.
Marcus Ranum@#20:
One of the places I worked for did, for a while, mapping of the orbits of space junk, using a big radio telescope as effectively an upward-pointing radar dish during times when it wasn’t being used for VLBI work. (Needless to say there were all sorts of political issues, from just making sure no airplanes were in the area when this thing lit up, to the fact that a lot of countries don’t want anybody that isn’t them tracking space junk because some of it might only be hiding as junk and still be spying in some fashion.)
One of the comments that I heard while we were doing this was that if you really wanted to completely trash a lot of things and probably large chunks of the world economy, it would be a lot of work, but not particularly complex work. Just boost a loose cloud of gravel up to the level of a geosynchronous orbit.
But going in the opposite direction.
Nobody’s done that sort of thing yet mostly because anybody capable of putting things into that high an orbit generally still wants to use it, and while this sort of action isn’t necessarily as bad as a full Kessler Syndrome situation (though there’s no guarantee it wouldn’t start one), at best it would still essentially render the single most specifically useful orbital line unusable for generations.
Researchers build brain-like memory device for AI sensors that may improve energy efficiency — phototransistor device combines light sensing, memory, and processing to cut data movement
The small (1.2mW, 16k sq ft) non-profit data center that I have worked in since it was built in 2001 has a closed loop glycol cooling system it was built that way on purpose, even though being in central Ohio means access to mass quantities of water is a non-issue.
A minor quibble, glycol is not a better conductor than water. A 40% glycol/water mix has about 95% the heat transfer capability of pure wate But other than that you have been paying attention to energy and thermodynamics! My specialty is energy engineering and while all of what you said is very doable, the return on investment is too long for the MBA types to be comfortable with so it always gets past over in favor of low 1st cost build techniques (cheap)..Increased energy costs could make this swing the other way but I don’t see our governments cutting energy subsidies for this type of user anytime soon.