Let me describe this in the most simple terms possible: You have speculators speculating about AI products. The speculators are not very smart when it comes to technology, and think RAM is RAM. There is at least three kinds of RAM that are important to this: DDR for system RAM, GDDR for GPUs, and HBM for high density enterprise products, and they are not interchangeable, there is no one-die-fits-all solution.
So, these speculators are like "oh no, more GPUs requires more RAM!", and then just start speculating on all RAM. Which of these RAMs are the ones that they need to worry about? Exclusively HBM, which is a minority in production, DDR and GDDR dominate production.
If you're into inference, and have older machines, you're buying Hxxx or Bxxx cards that use HBM, fit into dual slot x16 configurations, and you're jamming (optimally) 8 of them in. If you're into hardware that is newer, somewhere in the middle of the inference boom, you're using MXM cards. In either situation, the host machine has DDR, but if you're OpenAI, Anthropic, Microsoft, or Google, you're not building (more) inference machines like this.
The first two are buying Nvidia's all in one SBC solution: unified HBM, onboard ARM CPU to babysit the dual GPUs, has its own dual QSFP network controller that can RDMA, etc. No DDR or GDDR involved. Any machines built before this platform are being phased out entirely.
Microsoft is doing the same, but with AMD's products, the MI series that co-locates Epyc-grade Zen 4/5 CCDs with CDNA compute chiplets, running the entire thing off HBM, thus also unified and no DDR/GDDR needed. They, too, are phasing out machines older than this.
Google has a mix: they offer Nvidia all in one SBCs as part of GCP for legacy inference tasks (so your stack that can't run on AMD yet still can run), but also offer the same MI products that Microsoft offers via Azure's inference product, but also has their own TPUs that some of Gemini runs on; the TPUs run on HBM afiact. No DDR or GDDR here.
So, what does AMD or Intel do here? Lets say they waste fab time to make their own dies on the wrong process (TSMC and Intel-Foundry do not have for-RAM optimized processes)... they would be producing DDR and GDDR for a market that almost has its entire demand met. Intel lacks the die stacking technology required to build HBM, and TSMC I think can't do it for that many layers (HBM has 8 to 16 layers in current gen stuff iirc).
Micron, for example, already is bringing two large factories online here in the US to meet the projected growth in demand for the next 20+ years. When these factories finally start producing, it will not change the minds of speculators: they still seem to think AI datacenters need RAM, of any kind, and refuse to understand even the most basics of nuance. Also, when they come online, HBM will be a minority product; the AI inference boom is still just a bump in the road for them.
Nvidia kinda screwed their consumer partners, btw: they no longer bundle the GDDR required for the card with the purchase of the die. There is a slight short term bump in GDDR spot prices as partners are building up warchests to push series 60 GPUs into production, and once that is done, spot prices return to normal (outside of the wild speculation manipulation).
One last thing: what about LPDDR, used by AMD Strix Halo and Apple stuff? Speculation seems to have not actually effected it. I consider it as a sub-category of DDR (and some dies seem to work as either DDR or LPDDR as of DDR5, due to the merger of the specs by JEDEC), but since it isn't something you find in datacenters, it seems to have avoided speculation.
The Ryzen Max CPUs mentioned in the linked article? Uses LPDDR. Doubling down on the Ryzen Max product line might be a brilliant move.
> The speculators are not very smart when it comes to technology, and think RAM is RAM. There is at least three kinds of RAM that are important to this: DDR for system RAM, GDDR for GPUs, and HBM for high density enterprise products, and they are not interchangeable, there is no one-die-fits-all solution.
The commenter is also not very smart and does not realize companies making the RAM can trade capacity of one for another and any re-tooling at current price is still profitable.
The commenter also does not realize that is also true for lines currently making SSDs
They can trade capacity, but they generally aren't. The huge storage-only fabs owned by Samsung and Micron do runs that go for 9 months to 12 months.
Flash chips haven't been speculated on nearly as hard, and are suffering from the same sort of weird lack-of-nuance. Samsung, for example, isn't reassigning capacity to meet some sort of phantom datacenter demand that isn't already there, generically, across all datacenters, AI or not.
A lot of SSD price skyrocketing is largely "SSDs have RAM on them for cache", not "SSDs have flash chips, and they're both made at the same fabs"... which oddly effects low end SSDs that don't have external cache.
To make it worse, for the speculators who do understand this, because it isn't some universal homogeneous group, the flash chips that go into enterprise SSDs aren't the same that go into consumer SSDs.
The Big Three still aren't doing some major re-tasking of capacity, as the actual global demand isn't outstripping supply any more than normal. There is no short term problem to fix, speculators are just gonna have to stop hoarding toilet paper like its the start of Covid.
Edit: Oh, and if you want to ask how AMD/TSMC or Intel solve this? They can't, same reason why making their own in-house HBM isn't happening.
I'm glad Kioxia (formerly Toshiba) have been able to do that. However, I also know they've been having problems meeting demand for quite some time, and haven't been able to scale up nearly as fast as the big three have. There was an incident in 2019 and another in 2022 that killed entire runs of chips and screwed them during the Covid datacenter rush.
Micron killed Crucial because Crucial was a weird offering that competed with their own partners. This was always a weird problem, and it just didn't make financial sense to continue with it. One of the analyses I read was Crucial was less than 12% of sales.
Like, don't get me wrong, I've liked many Crucial products over the years, and even recommended some of them, but it was always weird they were trying to out-compete companies like Adata and other major ODMs.
The counterexample of this is Nvidia absolutely trying to kill their partners, and going to first party assembly and sales of products. Nvidia isn't even going to PNY anymore for ODM needs, but going directly to Foxconn.
Micron execs claiming its because of AI is a bit weird and revisionist, because they've been working on exiting the Crucial brand since long before they publicly announced it. The public didn't learn of any such plans until right before the Ballistix brand sunsetting was announced in 2021, but started years before that. Like, I know they're just playing to their shareholders, but its still a bit weird.
As far as I know, the current lineup is PNY still makes the workstation cards, possibly also the x16 server cards, but Foxconn is doing the Blackwell SBCs and MXMs, and those SBCs are a pretty big chunk of Nvidia's income right now. I also believe they have moved to Foxconn for the Founders Edition consumer cards.
Also, with the FEs, their partners are disallowed from making their own FEs, even if they make their own PCB from scratch and not based on any existing Nvidia design. Doesn't matter who makes the FE, it immediately puts partners at a great disadvantage if they can't make one too.
> So, these speculators are like "oh no, more GPUs requires more RAM!", and then just start speculating on all RAM.
Are you claiming that these speculators are buying DDR5 RAM and warehousing it somewhere? Or what exactly is the mechanism you are proposing here?
To me it seems much simpler - AI companies want HBM, but HBM and DDR5 share the same wafer production process and facilities, but the HBM process is much more fragile and takes three times the wafer production.
There isn't enough DDR5 RAM being produced, so prices go up.
There is no such thing as "consumer grade RAM". Servers still take DIMMs, ECC DIMMs just has more chips on it (previously 9 instead of 8, but now 10 instead of 8 as of DDR5; you'll see some DDR5 DIMMs with 5 instead of 4 because they're double die packages).
Micron, Samsung, and Hynix just basically sell you chips that comply with the JEDEC spec, and the DIMM manufacturers further bin them according to purpose. The highest end chips (that are stable at high clocks and acceptable voltages) end up in enthusiast performance products, the ones that don't work well at all but still meet JEDEC spec are sold to Dell/HP/Lenovo/etc for Grandma's Facebook machine, and the ones that are exceptionally stable at thermal design limits are plunked onto ECC DIMMs and sold to servers.
Also, as others have mentioned, its just a fab, and it can make any of the dies they're able to make. Whatever needs to be made to meet demand, they make, they just can't turn on a dime and react to quarterly concerns, and are locked into cycles that may range from 6 months to 18 months.
Side note that is also worth mentioning, sometimes you can order special bins of parts with features that wouldn't normally be available if you're willing to order enough. Recent example being Nvidia buying overclocked GDDR6 chips from Micron with additional features enabled; Micron was more than happy to become Nvidia's exclusive supplier for the custom GDDR chip if Nvidia was willing to buy out the entire run. Stuff like this happens every so often, but isn't the norm.
You just need an additional chip to move from "consumer grade" (ie no parity) to "server grade" (ie have parity). ECC support is actually in the memory controller which is in the CPU for the last 15 years. No magik.
The announcement means that they're closing Crucial - just like it says in the title and the first paragraph. The rest of that press release is outlining the mechanics of how that works + some fluff. Micron is going to continue producing the exact same memory chips in the exact same fabs. They're just not soldering it to a board, slapping the Crucial logo on it, and selling it directly to consumers. There's nothing stopping downstream vendors from buying Micron chips, soldering them to a board, and selling them to consumers as Micron was doing previously.
There's nothing in that press release that implies that the memory was somehow different (or "consumer-grade"). The _only_ thing they're saying is that they're ending their B2C business and focusing on B2B.
Didn’t you just describe the literal difference between consumer grade RAM that is soldered to a consumer format board vs a memory chip sold to a company to be soldered onto a product of the other company?
Calling it "consumer-grade RAM" is inaccurate - RAM is RAM. When you solder it to a board, you now have a DIMM that is carrying RAM chips. It's a semantic difference, but it's important.
So where are all these speculators storing DDR5, flash, and even spinning hard disks? Asking for a friend.
As a small buyer of all of those things supply at nearly any price has gotten very difficult to reliably predict week to week. When a lot of 100 64GB DDR5 sticks shows up available at a vendor, it’s at a take it or leave it price good for a couple hours. If I don’t pull the trigger they have another buyer for it and I might be waiting another month before anything becomes available again. We can no longer JIT for even failure replacement on our edge nodes.
Then you have the NVMe and even SATA SSD shortages. Still a bunch of very useful hardware out there I would love to find a decent deal on 8TB sata so I could repurpose it. Just doesn’t make any sense right now at current pricing and availability. Good luck trying to even find a batch of 12 of these disks at a time.
This goes for both enterprise and even prosumer I was willing to take for some of these uses.
Its mixed. Some of it really is Covid toilet paper behavior.
Datacenter customers, for example, have repair parts on hand; boxes of harddrives/ssds waiting to be put in, boxes of consumable parts, DIMMs waiting to replace ones that went faulty, entire machines already racked and waiting to take over for their fallen siblings, etc. Some of these customers added more to the spare parts pile. The big clouds manage their elastic demand of any sort of consumable or repair parts in volumes that are described in terms that fit cargo trucks in a quarterly basis, and they've already compensated.
Now, otoh, you have the truly psychotic people, that fill their basements with toilet paper, just hoarding more than they could ever use in their entire life. We've all seen that story where a guy was going to lose his house because he blew his mortgage money on toilet paper, and was selling it at a loss just to stay afloat. People like this exist in every crisis, and there's gonna be a headline in the near future where someone is gonna lose their house because they had like a hundred trays of DIMMs in their basement.
A few people I know who scrape eBay like its their job for electronics are just waiting for people to start fire-selling DIMMs and SSDs that got hoarded and they couldn't scalp people over; they're expecting half of MSRP or better sometime later this year.
> what about LPDDR, used by AMD Strix Halo and Apple stuff? Speculation seems to have not actually effected it
Good luck actually finding them on stock with 128GB+ RAM. I got strix laptop while ago, now price in EU is technically the same, but no stock. Maybe month or three
There is also claw hype. And large gwen3.5 models can run very well on DDR5 CPUs or mac minis...
Let me describe this in the most simple terms possible: You have speculators speculating about AI products. The speculators are not very smart when it comes to technology, and think RAM is RAM. There is at least three kinds of RAM that are important to this: DDR for system RAM, GDDR for GPUs, and HBM for high density enterprise products, and they are not interchangeable, there is no one-die-fits-all solution.
So, these speculators are like "oh no, more GPUs requires more RAM!", and then just start speculating on all RAM. Which of these RAMs are the ones that they need to worry about? Exclusively HBM, which is a minority in production, DDR and GDDR dominate production.
If you're into inference, and have older machines, you're buying Hxxx or Bxxx cards that use HBM, fit into dual slot x16 configurations, and you're jamming (optimally) 8 of them in. If you're into hardware that is newer, somewhere in the middle of the inference boom, you're using MXM cards. In either situation, the host machine has DDR, but if you're OpenAI, Anthropic, Microsoft, or Google, you're not building (more) inference machines like this.
The first two are buying Nvidia's all in one SBC solution: unified HBM, onboard ARM CPU to babysit the dual GPUs, has its own dual QSFP network controller that can RDMA, etc. No DDR or GDDR involved. Any machines built before this platform are being phased out entirely.
Microsoft is doing the same, but with AMD's products, the MI series that co-locates Epyc-grade Zen 4/5 CCDs with CDNA compute chiplets, running the entire thing off HBM, thus also unified and no DDR/GDDR needed. They, too, are phasing out machines older than this.
Google has a mix: they offer Nvidia all in one SBCs as part of GCP for legacy inference tasks (so your stack that can't run on AMD yet still can run), but also offer the same MI products that Microsoft offers via Azure's inference product, but also has their own TPUs that some of Gemini runs on; the TPUs run on HBM afiact. No DDR or GDDR here.
So, what does AMD or Intel do here? Lets say they waste fab time to make their own dies on the wrong process (TSMC and Intel-Foundry do not have for-RAM optimized processes)... they would be producing DDR and GDDR for a market that almost has its entire demand met. Intel lacks the die stacking technology required to build HBM, and TSMC I think can't do it for that many layers (HBM has 8 to 16 layers in current gen stuff iirc).
Micron, for example, already is bringing two large factories online here in the US to meet the projected growth in demand for the next 20+ years. When these factories finally start producing, it will not change the minds of speculators: they still seem to think AI datacenters need RAM, of any kind, and refuse to understand even the most basics of nuance. Also, when they come online, HBM will be a minority product; the AI inference boom is still just a bump in the road for them.
Nvidia kinda screwed their consumer partners, btw: they no longer bundle the GDDR required for the card with the purchase of the die. There is a slight short term bump in GDDR spot prices as partners are building up warchests to push series 60 GPUs into production, and once that is done, spot prices return to normal (outside of the wild speculation manipulation).
One last thing: what about LPDDR, used by AMD Strix Halo and Apple stuff? Speculation seems to have not actually effected it. I consider it as a sub-category of DDR (and some dies seem to work as either DDR or LPDDR as of DDR5, due to the merger of the specs by JEDEC), but since it isn't something you find in datacenters, it seems to have avoided speculation.
The Ryzen Max CPUs mentioned in the linked article? Uses LPDDR. Doubling down on the Ryzen Max product line might be a brilliant move.