Kaushal Trivedi comments on the ridiculous cost of LLMs.
The Jaw-dropping environmental impact of Large-Language-Models (LLMs)
Assuming static usage of 100 million weekly active users (ex. OpenAI chatGPT) and just 5 queries per user per week, the total energy consumption for operating an LLM like GPT-3.5 is staggering—around 44,200 MWh per year.
To put this in perspective:
With an emission intensity of 0.4 kg of CO2 per kWh, this level of energy consumption emits the same amount of CO2 as making 15,000 round trips in your petrol-powered car from Kashmir to Kanyakumari (a road distance of 3,600 km one-way).That’s massive!
As we continue to innovate, it’s crucial to consider the environmental impact and strive for more sustainable solutions.
Because at the end, there’s no PLANet B.
First let me say I would love to make that trip once. Not 15,000 times, but once would be awesome. I’d see a lot, learn a lot.
What do I get from ChatGPT for that massive investment? A lot of companies get to slap “AI” on their products, and as near as I can tell, it only makes them worse. Is Google improved by adding AI to their front end? No. What would improve Google is stripping off the layers of cruft they’ve accumulated and monetized over the years, simplifying the algorithms, and making internet search simple and streamlined again. Who thinks a phone book benefits from a half-assed chat function?
I guess if you’re selling car rentals in Kanyakumari, it’s profitable to get people to make repetitive road trips.
Hey, can I make a detour to Bengal? Maybe make some scenic stops?
The truly sad, and aggravating, things about the LLM problem is that no one among legislators, or anyone doing threat assessments, is seeming to give a damn about the waste of resources. Nope.. They are worried that a) they will somehow get good enough to write believable scripts (they won’t, but the industry already pays for crap all the damn time anyway, so it would hardly change much – media execs have always been idiots, with no clue what the public wants, or actually likes), b) they might “take over”, despite the fact that these freaking things can’t even do the basic thinking needed to ask themselves, “Did the thing I just came up with make sense, or did I ‘hallucinate it’?”, and c) half the idiots out there confuse ChatGPT, which requires massive server farms to run, with image generators, which you can literally install and run on your own home computer (and doesn’t as I understand it, in such configuration, require that it be connected to a bloody server farm).
Basically, they are worried about things that can’t happen, won’t happen, are only a problem because corporate CEOs are idiots, and have no clue what the limitations are, but love them some short cuts, or are utterly imaginary (perhaps even a hallucination? lol). The real problem, of course, is freaking energy overuse to run them – which means that if half the crazies in politics where not scared to death of the tech, for no sane reason, they would probably be trying to protect its use, like it was fossil fuels.
Seriously… How the F did we end up in this timeline?
Cost benefit-analysis? As in “We reap the benefits, everybody else pays the costs”? Yup, looks like the LLM craze is business as usual.
Its pretty durn warm in India right now :
Source : https://www.abc.net.au/news/2024-05-30/india-delhi-heatwave-records-temperature-climate-change/103911178
Even Nepal at our planet’s third Himalayan pole is experiencing record heat for it – as is so very much of our planet.
Wikipedia: “The 2021 world total energy production of 14,800 MToe corresponds to a little over 172 PWh / year, or about 19.6 TW of power generation.”
172 PWh = 172,000,000,000 MWh.
44,200 MWh = a very, very tiny proportion of that.
(How much energy is used for luxury cruise liners?
Bulk oil tankers?)
(How much to run porn sites?)
Those two other things are useful, John. :)
I have read repeatedly that one of the drawbacks to crypto-currencies is the huge amount of energy they require to create keys. I wonder how much the recent surge in LLM technology is related to crypto.
John Morales @ #5 — It was established decades ago that the Internet owes a lot to porn. There are upsides and downsides.
jo1storm @ #6 — LLMs and AI in general have numerous benefits for science, health, and economy monitoring. A lot of science these days…say climate science…involves processing enormous amounts of data. Direct human analysis is not scalable. However, writing term papers for school students is not one of the main benefits.
@robro: the relation between LLMs and cryptocurrency is … nothing at all. I mean, apart from both being the product of centuries of scientific and mathematical research and societal development.
In addition to not actually being a lower of energy compared to what humanity is using, the emissions intensity estimate there is the US average. Most of the power use for data centres is quite a lot lower intensity, because they seek out low-carbon power out of the goodness of their … wallets. Low-carbon power is cheaper.
Is it though? I’m not going to say that overuse of LLMs, or energy usage at data centers in general, isn’t a problem, but the way the numbers are being present there is kind of suspicious. Sure, driving back and forth across India 15,000 times seems weird, but why would it make sense to compare 100 million people using LLMs, to one guy’s driving? Or even 15,000 people’s driving?
Well, we can fix that. Using their own numbers, 23,60015,000 = 108,000,000 kilometers traveled in total. So let’s divide that by 100,000,000 people instead of 1…
…And we see that using an LLM 5 times is equivalent to driving just over a kilometer.
(Which to be fair, is pretty expensive. Transportation energy usage is still the larger problem: I’d bet most people in the US at least drive a lot more than 1km, and use LLMs rather less. Nevertheless, if someone uses LLMs dozens or hundreds of times per week, that energy usage could be on the same order as their transportation usage.
But also, 0.4 kg of CO2 per kWh is around the current US average for electricity production. Since LLMs, unlike cars, are already fully electrified, that number will likely only get better as more of the grid is converted to renewables.)
(Ugh – the * formatting always gets me. That should read 2 * 3,600 * 15,000 = 108,000,000 kilometers.)
These numbers are a bit misleading: 44,200 MWh sounds like a lot, but that is the annual production of 3-4 typical wind turbines. For 100,000,000 users, that is not actually a lot of power, relatively speaking.
We shouldn’t ignore the energy cost it takes for a human to do these tasks for themselves on a computer with a monitor or two. By his numbers, a query is about 0.0018 kWh, about the amount of energy it takes to run a computer for 30-60 seconds. If 5 queries in a day can shave off five minutes of writing, coding, etc. on your computer during that day, there would be a net reduction in energy consumption. Granted, there is a lot of spurious use of these AI tools, but there is a lot of writing and coding where these tools are more efficient than humans.
@#5, John Morales:
Relative to AI — or for that matter the whole Cryptocrap thing — very little, and no more than any equivalent non-pornographic media (text/images/video). Apache doesn’t magically become less efficient when answering a request for a picture of breasts instead of a cat macro, Firefox doesn’t magically burn more CPU cycles for Pornhub than for YouTube, and your ISP’s router doesn’t suddenly melt its heat sinks because the packets passing to you make up a smutty fanfic instead of stock market figures.
I think you are wrong on the basis of scale, and I know you’re ignoring the power use of the client machines (the users’). One imagines porn users spend a bit more time watching their porn than AI users spend making queries.
So. Got any actual, you know, numbers to support your claim?
(Oh, and of course, AI porn!)
Using energy is not the main problem. The way the energy is generated is the biggy.
In Oz, on a good day, a couple of our cities are capable of 90% renewable energy.
I realize that many here are seeking every possible negative thing they can find about LLMs, but the numbers quoted in the article, far from doom and gloom, actually point out the LLM as being rather power efficient. Multiplying everything out across those 100 million users means that, in the end, ChatGPT consumes about 50 mW per user. That’s like each user adding an LED status indicator to their existing electricity usage. As pointed out above, it is an almost infinitesimal portion of the world energy economy.
Now, cryptocurrency, on the other hand…
Yeah, its a bit weird they are focused on “power use”, given that the real problem that pretty much everyone else has pointed out with both crypto and LLM servers is, “Cooling by using massive amounts of water, which we can’t then use for drinking, during what is increasingly closer and closer to droughts.” I hate how our current reality is making me slightly paranoid (for example, isn’t it odd that the same time that energy companies are trying to offer Trump a bribe almost every other company is raising prices, especially in red and swing states, even while the economy is far better than it has been for ages, and it just happens to perfectly correspond to the same election?), but one almost has to wonder if the “power usage” BS is a distraction, to get people to ignore the water use problem…