Rendered at 18:45:02 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
vannevar 18 hours ago [-]
AI compute is not the new oil. Demand for oil is very inelastic: it's very difficult to increase the thermodynamic efficiency of the physical processes that convert oil into energy. AI compute efficiency, on the other hand, is improving exponentially. AI will end up being just slightly more expensive than the raw electricity required to produce it.
Data center demand right now has much more to do with private credit's demand for new deals, since the binge they went on around the pandemic led to the accumulation of a lot of bad debt and depreciating assets. Data centers are a tool for extracting money from insurance trusts and pension funds to prop up private credit and create debt-driven artificial "revenue" for the hyperscalers and Nvidia.
vatsachak 18 hours ago [-]
Yeah AI is electricity, not oil. It's boring but it's already true.
I'm building a side business and decided to go with open router because that's what AI is going to look like in the future. Just intelligence/$ everything else kind of not important.
Unless, they solve the issue of model output diversity and Claude becomes very different from ChatGPT
solyin 13 hours ago [-]
AI is a broader field than ChatGPT and Claude. Waymo is AI. Suno is AI. TikTok's recommendation algorithm is AI, Protein folding, Weather predictions. I can keep going.
Sure, Claude and ChatGPT are electricity, but oil is not just gasoline and diesel. It's also responsible for the colors all over our world. Plastic comes from oil. So chatbots might be electricity but the broader field of AI seems more like oil to me.
AgentMasterRace 14 hours ago [-]
you're getting robbed on the fee's
vatsachak 13 hours ago [-]
I mean I route everything to Luna so it's really cheap. Plus if Luna isn't doing too hot I can go to Haiku easily
aleqs 2 hours ago [-]
If you 'route everything to luna' via OpenRouter then you're literally paying 10-20x the subscription rate
aleqs 13 hours ago [-]
pretty much any sub is far better usage/intelligence per $ than openrouter, opencode go is the best one I know of at the moment
vatsachak 11 hours ago [-]
OpenRouter is great because it's super easy to add keys and track spend. Plus it comes with a good API for tracking user token spend.
I mean the average Luna cost is so low that it would be eaten up by a dollar per month in my low intensity use case.
aleqs 3 hours ago [-]
Is your purpose to market OpenRouter and OpenAI/Anthropic?
what you're suggesting is like 10x the cost of other options.
EGreg 18 hours ago [-]
[flagged]
simoncion 17 hours ago [-]
> AI will end up being just slightly more expensive than the raw electricity required to produce it.
AFAICT, this hasn't happened for any "cloud" compute company that has been in business for -say- at least five years. Unless your prediction is that roughly no one will use "cloud" "AI", I'm rating this prediction as "totally bogus". And if that is your prediction, then what you're saying is
Assuming that you're not doing anything that needs exotic system administrators, nearly all of the cost of computing when you own your own hardware is the power for that hardware.
which... unless you're buying used hardware, I don't think that's true, either. Hetzner quotes a max of 167.23 EUR/month for a full rack and what I think is 2400W of power, and lists the rate for Finnish power as 0.22 EUR per kWh. [0] If we assume that Hetzner's spending cap doesn't apply and we are forced to pay the full quoted rate for power in Finland, I think that works out to a bit under 4.6K EUR per year:
The last time I got quotes for most of a rack of server hardware was like a decade ago, and that pretty-bog-standard server hardware ran around 200K USD. If we assume a ten-year expected life [1], that hardware cost about 17K EUR per year. Assuming I haven't fucked up my math, [2][3] the cost of the hardware here is more than three times the cost of power.
Note that I'm using roughly decade-old hardware prices and allegedly-current electricity prices. If we used hardware prices from two years ago -or worse, from today- the hardware would cost far more per year.
[1] ...which I think is between two and three times longer than is standard to expect in the business world...
[2] ...it's very likely that I have! Corrections are definitely welcome.
[3] ...it's also possible that USians get gouged for server hardware and that it's like a quarter of the cost if you buy it in Finland. I expect that there's definitely a price difference, but I doubt that it's very large.
vannevar 14 hours ago [-]
>AFAICT, this hasn't happened for any "cloud" compute company that has been in business for -say- at least five years.
Most cloud software has had moats. AI does not. And AI is eating software, which means those historical margins are going away for a lot of cloud software,too. That's what the SAAS-pocalypse is about.
>Assuming that you're not doing anything that needs exotic system administrators
Remember, these are also going to be AI.
>Assuming I haven't fucked up my math, [2][3] the cost of the hardware here is more than three times the cost of power.
This is a fair point, but it just means the COGS is higher, not that the margin is any better. I stand by the prediction that AI margins will be very slim.
simoncion 13 hours ago [-]
> I stand by the prediction that AI margins will be very slim.
This I don't disagree with... but you and I probably believe this for very different reasons. However, the rest of what you've said -in this comment and the first one I replied to- is just bonkers word salad.
Tanjreeve 14 hours ago [-]
Even if that scenario did happen (I agree I don't think most of the major providers would survive on a commodity utility model). Most of the advances in AI are driven by huge amounts of funding for research. If everyone goes to open source models/utility based pricing etc then it'll be cheaper but the current rate of improvements are being overwhelmingly driven by the frontier labs (for better or worse) and they're being driven by vast amounts of VC funding predicated on making vast amounts of money.
I'm very dubious we can have our cake and eat it of constant magnitude order improvements and also commodified utility pricing.
vatsachak 16 hours ago [-]
What? You're saying that Hetzner is losing money?
168 * 12 < 4562
simoncion 14 hours ago [-]
168 * 12 < 4562
That is correct. It's a little under half that. It's why I said
If we assume that Hetzner's spending cap *doesn't* apply...
> You're saying that Hetzner is losing money?
If I were saying that, I would have said it. Unless "XXX max/mo." doesn't mean what it means in plain English, I _very_ strongly expect that it's very rare for colo customers to suck down all available power 24/7 and that Hetzner has a Stern Talk with customers who do but didn't disclose their intent to do so up front. I mean, you tell me... if you've ever lived in an apartment where your power and water was included in the rent and was metered with a whole-building meter (rather than per-unit meters), did you consume every amp and gallon that could be delivered to you, or did you use just a bit more than what was useful to you in the course of everyday life? [0]
I worked the math the way I did (even doubling or tripling the expected lifetime of the server hardware) to make the numbers as favorable for the "electricity is almost 100% of the cost of computing" side of the argument as they could reasonably be.
Are you saying that I fucked up my math?
[0] And if you're never lived in such an arrangement, ponder instead how you would behave if you did find yourself in one.
vatsachak 14 hours ago [-]
Fair. Your argument is that the price of the hardware dominates the price of electricity in cloud computing
simoncion 13 hours ago [-]
> Your argument is that the price of the hardware dominates the price of electricity in cloud computing.
No. The numbers indicate that the price of all "full-sized" computing is mostly the cost of hardware, [0] regardless of whether that's "cloud" computing, "on prem" computing, or -er- "colocated" computing.
And by "full-sized" I mean "computing with regular PC or server-type hardware". I'm sure someone could build some absurdly cheap exotic stuff that invalidates all this math, but I don't expect anything that will be found in a datacenter, server room, or on someone's desk at work and whose primary job is doing compute that customers are paying for to be that stuff.
[0] Unless that hardware is acquired at a _deep_ discount... which usually isn't possible if you're buying new hardware. And if you're buying used hardware at a deep discount, that usually means that someone paid a normal price for that hardware at some point.
zekrioca 14 hours ago [-]
They probably meant that AI = fossil fuels, because gas is the energy source from where majority of electricity for AI datacenters will (proportionally) come from.
redanddead 13 hours ago [-]
does jevon's paradox negate this, i think yes, demand is growing and scaling is the only real path forward
>create debt-driven artificial "revenue" for the hyperscalers and Nvidia.
what for? Cui bono?
piva00 13 hours ago [-]
The new oil is data, and has been steadily growing into this position for the past 20 years.
LLM are just now consuming data, ad networks have been gobbling on it for many years.
vinyl7 17 hours ago [-]
History has proven that businesses will relentlessly and needlessly raise prices to increase revenue quarter over quarter.
phpnodejspwn 16 hours ago [-]
[flagged]
nashashmi 15 hours ago [-]
More reasons for regulation.
AI companies, LLM makers, AI solution providers, and AI users should be inexcusably responsible for any and all harms caused by their misaligned automations across the chain. This alone will encourage self regulation and fix a broken industry that has too long prospered on fail fast and break things to be more considerate of their actions.
The second form of regulation should be on excessive energy consumption. The US Department of Energy needs to declare energy to be a vital human resource, and exorbitant uses for computing to be curtailed. Bidding on energy prices should be restricted to the excess markets where free market dictates price. And supply will reduce availability.
Long term, this will put the right kind of pressures and halting of demand. It will slow down investment in infrastructure and increase investment in research and development. It will prioritize efficient AI uses over low quality trial and error.
visarga 7 hours ago [-]
> The second form of regulation should be on excessive energy consumption. The US Department of Energy needs to declare energy to be a vital human resource, and exorbitant uses for computing to be curtailed.
If you allow AI to use as much as it needs we get much higher capacity and cheaper energy down the line. Growing pains.
I see AI like the coming of a baby in the family - suddenly you need to get a larger house, baby room, ensure babysitting and education, an effort for the whole family. This is the initial cost, which is high for both a baby and AI.
nashashmi 3 hours ago [-]
Yes there is a real possibility that AI will reduce energy consumption. But you can have both: curtailed AI usage + Energy-focused AI usage. Otherwise we are risking “short term pains” for future promises.
pseudosaid 2 hours ago [-]
yeah this is so shirtless. Why is the industry trying act babies dont take time to incubate. side stepping these will lead to a still born. This is not the initial cost and it's entirely self-directed. This is entirely avoidable, in fact they need to do this as quickly as possible to push an accelerationist agenda.
kamban 14 hours ago [-]
Fair point. I guess very few have the incentive to do this voluntarily.
jordemort 16 hours ago [-]
the new oil as in we’re going to make a bunch of products that are needlessly packaged with derivatives of it, leaving behind a toxic
mess that will take generations to clean up?
DeluluDon 15 hours ago [-]
It's already happening with development smartwatches.
Avicebron 18 hours ago [-]
AI has to take us where We want to go before its compute can be considered on par with oil.
userbinator 17 hours ago [-]
I think that's starting to happen. There's a new vibecoded project shown off here almost every day. YouTube has gotten a lot more entertainment rising above the slop. People are beginning to realise what AI can do for them.
byzantinegene 16 hours ago [-]
i would argue that youtube has gotten alot worse this year, and every vibecoded project shown off here has been pretty soulless and pointless.
creata 16 hours ago [-]
It's hard to say that all the (to varying degrees) vibecoded security vulnerabilities posted to the frontpage are pointless.
creata 17 hours ago [-]
> There's a new vibecoded project shown off here almost every day.
Every other post on the News.YC frontpage has been about AI for the past few years. That means very little.
> YouTube has gotten a lot more entertainment rising above the slop.
Examples would be great. I'm looking on YouTube's "hot" playlists and haven't yet seen anything marked AI, although that of course doesn't rule out the use of AI.
userbinator 13 hours ago [-]
People are vibecoding and vibemodding software like never before. https://news.ycombinator.com/item?id=50196718 is on the front page as I write this, and there have been many others sharing their story of how AI fixed bugs in software they didn't have the source to, by patching the binary.
The thing about AI entertainment is that it's far more targeted and niche, so you're unlikely to find huge central gatherings around one popular thing, but instead it's spread out over many smaller channels.
creata 13 hours ago [-]
Sorry, when you said "People are beginning to realise what AI can do for them.", I got the impression you were referring to the general public lessening their visceral hatred of anything AI-generated, not niche groups of people doing very impressive things.
DenisM 15 hours ago [-]
This caught my eye:
> Architect Financial Technologies, founded in 2023, offers futures in compute linked to the value of chips in America. The firm has also recently launched a platform where model-makers bid to provide “inference”—the useful output produced by an AI model in response to a prompt—rather than the underlying compute capacity.
I think the idea of selling inference is solid, but I’m not convinced Wall Street is going to be doing that. Superscalers are better positioned.
Inference will likely be priced by evals, specific for each workload.
raziel2701 17 hours ago [-]
I remember in the 2010s when people were saying Big Data is the new oil
16 hours ago [-]
metalspot 13 hours ago [-]
But they were right? For good or ill data and digital services are more valuable and just as ubiquitous as oil.
kylehotchkiss 17 hours ago [-]
Remember when people thought crypto and NFTs were the new oil... lol. LLMs will have staying power, there's just a day where this seems like it's all going to run on our laptops. OpenAI's fear of that and tanking the RAM market was just a temporary setback.
Tanjreeve 14 hours ago [-]
Oils probably a poor metaphor but between the ubiquity of companies deriving significant value from data + analytics on larger than memory datasets and the recent mad rush for training data I don't think that is the wildest tech prediction ever.
I think LLMs are more likely to end up as something boring we don't think about like cloud computing but I don't think comparisons with crypto etc are fair as it's a much more generalisable technology.
j16sdiz 17 hours ago [-]
infrastructure project big enough to affect qol in neighborhood: checked
corruption: checked
regulatory capture: in progress
Ya, the new oil.
daft_pink 7 hours ago [-]
Moore's law is going to eventually eviscerate the compute needs and allow us to run compute locally and cheaply.
Good4boothee 7 hours ago [-]
It depends on exact wording but in general Moore's law has been dead for few years. Compute (CPU/GPU) continues to squeeze more performance with better architecture, but RAM and NAND(SSDs) are already as simple as possible and only improvements in density come from stacking multiple layers.
the concern is real because the current market is treating AI as "hand off and controlling all" agent, which is not efficient and not desirable because AI is still not smart enough to take on this initiative yet, and most of the work (especially repeatable ones don't need full AI. I would guess 5-10% of our current spending is actually usefully achieving some positive outcomes, the rest are just wasted. But the global resources pouring into AI field is the real trend.
pier25 15 hours ago [-]
We’re trying to get rid of fossil fuels…
18 hours ago [-]
danielmarkbruce 14 hours ago [-]
This seems silly. It's analogous to an electricity market where every megawatt is a different voltage, in a different city with no transmission lines... might stop working halfway through the job, and only works properly if you buy another 500 megawatts from the same supplier... good luck getting this market off the ground.
Data center demand right now has much more to do with private credit's demand for new deals, since the binge they went on around the pandemic led to the accumulation of a lot of bad debt and depreciating assets. Data centers are a tool for extracting money from insurance trusts and pension funds to prop up private credit and create debt-driven artificial "revenue" for the hyperscalers and Nvidia.
I'm building a side business and decided to go with open router because that's what AI is going to look like in the future. Just intelligence/$ everything else kind of not important.
Unless, they solve the issue of model output diversity and Claude becomes very different from ChatGPT
Sure, Claude and ChatGPT are electricity, but oil is not just gasoline and diesel. It's also responsible for the colors all over our world. Plastic comes from oil. So chatbots might be electricity but the broader field of AI seems more like oil to me.
I mean the average Luna cost is so low that it would be eaten up by a dollar per month in my low intensity use case.
what you're suggesting is like 10x the cost of other options.
AFAICT, this hasn't happened for any "cloud" compute company that has been in business for -say- at least five years. Unless your prediction is that roughly no one will use "cloud" "AI", I'm rating this prediction as "totally bogus". And if that is your prediction, then what you're saying is
which... unless you're buying used hardware, I don't think that's true, either. Hetzner quotes a max of 167.23 EUR/month for a full rack and what I think is 2400W of power, and lists the rate for Finnish power as 0.22 EUR per kWh. [0] If we assume that Hetzner's spending cap doesn't apply and we are forced to pay the full quoted rate for power in Finland, I think that works out to a bit under 4.6K EUR per year: The last time I got quotes for most of a rack of server hardware was like a decade ago, and that pretty-bog-standard server hardware ran around 200K USD. If we assume a ten-year expected life [1], that hardware cost about 17K EUR per year. Assuming I haven't fucked up my math, [2][3] the cost of the hardware here is more than three times the cost of power.Note that I'm using roughly decade-old hardware prices and allegedly-current electricity prices. If we used hardware prices from two years ago -or worse, from today- the hardware would cost far more per year.
[0] <https://www.hetzner.com/colocation/>
[1] ...which I think is between two and three times longer than is standard to expect in the business world...
[2] ...it's very likely that I have! Corrections are definitely welcome.
[3] ...it's also possible that USians get gouged for server hardware and that it's like a quarter of the cost if you buy it in Finland. I expect that there's definitely a price difference, but I doubt that it's very large.
Most cloud software has had moats. AI does not. And AI is eating software, which means those historical margins are going away for a lot of cloud software,too. That's what the SAAS-pocalypse is about.
>Assuming that you're not doing anything that needs exotic system administrators
Remember, these are also going to be AI.
>Assuming I haven't fucked up my math, [2][3] the cost of the hardware here is more than three times the cost of power.
This is a fair point, but it just means the COGS is higher, not that the margin is any better. I stand by the prediction that AI margins will be very slim.
This I don't disagree with... but you and I probably believe this for very different reasons. However, the rest of what you've said -in this comment and the first one I replied to- is just bonkers word salad.
I'm very dubious we can have our cake and eat it of constant magnitude order improvements and also commodified utility pricing.
168 * 12 < 4562
If I were saying that, I would have said it. Unless "XXX max/mo." doesn't mean what it means in plain English, I _very_ strongly expect that it's very rare for colo customers to suck down all available power 24/7 and that Hetzner has a Stern Talk with customers who do but didn't disclose their intent to do so up front. I mean, you tell me... if you've ever lived in an apartment where your power and water was included in the rent and was metered with a whole-building meter (rather than per-unit meters), did you consume every amp and gallon that could be delivered to you, or did you use just a bit more than what was useful to you in the course of everyday life? [0]
I worked the math the way I did (even doubling or tripling the expected lifetime of the server hardware) to make the numbers as favorable for the "electricity is almost 100% of the cost of computing" side of the argument as they could reasonably be.
Are you saying that I fucked up my math?
[0] And if you're never lived in such an arrangement, ponder instead how you would behave if you did find yourself in one.
No. The numbers indicate that the price of all "full-sized" computing is mostly the cost of hardware, [0] regardless of whether that's "cloud" computing, "on prem" computing, or -er- "colocated" computing.
And by "full-sized" I mean "computing with regular PC or server-type hardware". I'm sure someone could build some absurdly cheap exotic stuff that invalidates all this math, but I don't expect anything that will be found in a datacenter, server room, or on someone's desk at work and whose primary job is doing compute that customers are paying for to be that stuff.
[0] Unless that hardware is acquired at a _deep_ discount... which usually isn't possible if you're buying new hardware. And if you're buying used hardware at a deep discount, that usually means that someone paid a normal price for that hardware at some point.
>create debt-driven artificial "revenue" for the hyperscalers and Nvidia.
what for? Cui bono?
LLM are just now consuming data, ad networks have been gobbling on it for many years.
AI companies, LLM makers, AI solution providers, and AI users should be inexcusably responsible for any and all harms caused by their misaligned automations across the chain. This alone will encourage self regulation and fix a broken industry that has too long prospered on fail fast and break things to be more considerate of their actions.
The second form of regulation should be on excessive energy consumption. The US Department of Energy needs to declare energy to be a vital human resource, and exorbitant uses for computing to be curtailed. Bidding on energy prices should be restricted to the excess markets where free market dictates price. And supply will reduce availability.
Long term, this will put the right kind of pressures and halting of demand. It will slow down investment in infrastructure and increase investment in research and development. It will prioritize efficient AI uses over low quality trial and error.
If you allow AI to use as much as it needs we get much higher capacity and cheaper energy down the line. Growing pains.
I see AI like the coming of a baby in the family - suddenly you need to get a larger house, baby room, ensure babysitting and education, an effort for the whole family. This is the initial cost, which is high for both a baby and AI.
Every other post on the News.YC frontpage has been about AI for the past few years. That means very little.
> YouTube has gotten a lot more entertainment rising above the slop.
Examples would be great. I'm looking on YouTube's "hot" playlists and haven't yet seen anything marked AI, although that of course doesn't rule out the use of AI.
The thing about AI entertainment is that it's far more targeted and niche, so you're unlikely to find huge central gatherings around one popular thing, but instead it's spread out over many smaller channels.
> Architect Financial Technologies, founded in 2023, offers futures in compute linked to the value of chips in America. The firm has also recently launched a platform where model-makers bid to provide “inference”—the useful output produced by an AI model in response to a prompt—rather than the underlying compute capacity.
I think the idea of selling inference is solid, but I’m not convinced Wall Street is going to be doing that. Superscalers are better positioned.
Inference will likely be priced by evals, specific for each workload.
I think LLMs are more likely to end up as something boring we don't think about like cloud computing but I don't think comparisons with crypto etc are fair as it's a much more generalisable technology.
corruption: checked
regulatory capture: in progress
Ya, the new oil.
But checkout https://thegrid.ai/
https://hnnoai.readpipe.org/