From "Computing Power" to "Storage Capacity": In the Second Half of the AI Hardware Race, Who Is Struggling?
Hello everyone, I'm your financial journalist and economist.
If you've been following the tech or stock market recently, you might have noticed an interesting shift: in the past, the focus on AI was all about NVIDIA's GPUs (graphics cards), which were seen as the "brains" of the system. But in the second half of 2025, the focus has shifted to storage devices—DRAM, NAND, and hard drives—once considered secondary components.
Why? Because storage capacity has become the new bottleneck.
Simply put, the problem is no longer about the speed of computation but about the amount of data that can be stored and the speed at which it can be retrieved. It's like a skilled chef (the GPU); no matter how fast they are, if the ingredients (data) can't be retrieved quickly from the "fridge" (memory), the chef can't work efficiently.
Today, we'll break down this in simple terms and explore this in-depth report on HBM (High Bandwidth Memory) to understand what's driving this "storage capacity" revolution and the business logic and investment opportunities behind it.
---
Why Does AI Suddenly Require So Much Memory?
Many people have a misconception that AI is just about doing calculations and has nothing to do with storage. This is completely wrong. AI's demand for memory is not growing linearly but is exponentially increasing. This is mainly due to three factors:
1. Growing Model Sizes (Parameter Storage)
You can think of large AI models as extremely complex dictionaries. These dictionaries used to contain only a few million entries but have now grown to trillions.
- Data Trends: For example, the parameter size of Qwen3.8 Max has increased by more than 30 times compared to its first version.
- Consequences: Just storing this dictionary alone requires more memory—from 135GB to 2300GB (about 2.3TB). And that's before the model even starts working!
2. Longer Conversational Memories (KV Caches)
This is often overlooked but is a major memory consumer. The more you talk to an AI, the more context it needs to remember.
- Example: If you have a 100-page conversation with an AI, it needs to remember all that content to respond appropriately.
- Current Situation: Although software has improved memory compression, modern AI models can handle contexts of hundreds of thousands or even millions of words.
- Combined Effect: If 32 people are talking to the AI simultaneously, the combined memory requirement for these conversations alone is 1.3TB, plus the 2.3TB for the model, totaling 3.6TB!
3. AI Agents' Additional Tasks
AI agents don't just answer questions; they also code, operate Excel, and send emails. The temporary files, logs, and intermediate data generated by these tasks consume significant amounts of memory, even if they don't require the highest-end HBM.
In summary, speed of computation is still important, but the ability to store and retrieve data quickly has become equally, if not more, crucial.
---
Why Is HBM the New Favorite? Can't We Use Ordinary Memory?
If memory is so important, why not just use the DDR5 memory we have in our computers? The answer lies in two key metrics: capacity and speed.
1. Physical Limits of Ordinary Memory
- Manufacturing Challenges: It's difficult to shrink memory chips to 2nm or 3nm like CPUs. Physical limitations make this impossible, and current memory technology is stuck at 11-12nm, with potential improvements to 10nm in the next few years.
- Limited Space: Since individual memory chips can't be made very small, more chips are needed. However, the space around the GPU is extremely valuable, and memory modules can't be placed as simply as on a regular motherboard.
2. HBM's Innovative Technology: Stackable Memory
HBM's innovation lies in its ability to stack multiple layers of memory chips vertically instead of horizontally.
- Vertical Stacking: Imagine ordinary memory as single-story houses; HBM is like a multi-story parking garage built next to the GPU. It stacks 12 or even 16 layers of chips together.
- Silicon Through-Vias (TSVs): Engineers create thousands of tiny vertical holes in the chips and fill them with copper to allow signals to travel directly between layers.
- Result: HBM can fit more data in the same area than ordinary memory.
3. Why Faster Speed?
- Bit Width: Ordinary memory transfers 32 bits at a time, while HBM transfers 1024 or even 2048 bits. It's like comparing a single-lane road to a 1000-lane highway.
- Shorter Distances: Data is transferred directly between the GPU and HBM, reducing latency significantly.
In short, HBM is designed to solve the problems of limited space and speed.
---
Technical Challenges and High Costs of HBM
Despite its advantages, HBM is extremely difficult to manufacture, which is why it's several times more expensive than DDR5. The main challenges include:
1. Stackable Fabrication
- Punching Holes: Creating thousands of precise holes in thin silicon wafers is extremely challenging, requiring precise cooling and electrical connections.
- Bonding Chips: How do you bond 12 layers of chips together? SK Hynix uses the MR-MUF (Reflow Soldering + Liquid Resin) process, which is efficient and reliable. Samsung and Micron use the TC-NCF (Thermal Compression + Non-Conductive Film) process, which is more complex and less reliable. SK Hynix has an advantage here.
- Competitive Landscape: SK Hynix dominates HBM due to its MR-MUF process, while Samsung and Micron are working on next-generation bonding technologies.
2. Packaging
HBM must be packaged with the GPU using a special silicon interposer. The precision required for the interposer's circuits is so high that only TSMC's CoWoS (Co-Packaged on-Wafer Substrate) technology can meet the standards.
3. Base Die Customization
The base layer of HBM is no longer standard; it must be customized for specific customers like NVIDIA. For example, NVIDIA's NVHBM uses proprietary protocols to improve bandwidth and reduce power consumption. This means HBM is no longer a standardized product but a semi-customized component.
---
Changing Industry Dynamics
This HBM revolution has significantly altered the memory industry:
1. Reduced Independence of Memory Manufacturers
Memory manufacturers (Samsung, SK Hynix, Micron) used to control the entire process from design to production. Now, the chain is more fragmented:
- Memory Manufacturers: Produce memory chips and stack them.
- TSMC: Manufactures the base layer and performs the final packaging.
- End Customers (e.g., NVIDIA: Participate in the design of the base layer and define the specifications.
Consequences: Memory manufacturers' influence has weakened. If NVIDIA changes suppliers or TSMC's production capacity is limited, they can't meet demand.
Value Shift
- Short Term: The AI boom has expanded the market, benefiting all parties. SK Hynix and Micron's stock prices have risen significantly.
- Long Term: As customization deepens, memory manufacturers may become mere contract manufacturers, with value shifting to manufacturers of packaging technologies (TSMC) and end customers with design control.
Investment Logic
- Past: Investing in memory stocks focused on price cycles.
- Now: Investors look at technical barriers (who has the best stacking technology?) and production synergy (who gets access to TSMC's packaging capabilities?).
- SK Hynix: Benefits from its MR-MUF process and close relationship with NVIDIA.
- Samsung/Micron: Catching up but facing challenges with process transitions and patent issues.
- TSMC: As the packaging and base layer provider, TSMC's position is strengthened.
Future Trends
The article outlines several future developments:
1. Increasing Stack Layers: More layers will increase capacity but also raise manufacturing difficulties and costs.
2. Customization: HBM will become more like custom chips rather than standard memory modules, with different versions for various customers.
3. Balancing Cost and Performance: HBM's high cost limits AI adoption. There may be efforts to find a balance between HBM and cheaper DDR5 or to reduce costs through technological improvements.
4. NAND Flash Memory: Although HBM is the focus, NAND flash memory (used for storage) is also seeing significant demand due to the explosion in AI data.
Conclusion
In the second half of 2025, we're witnessing more than just a rise in memory prices; we're seeing a fundamental shift in AI hardware architecture. Computing power is the engine, but storage capacity is the fuel and the pipelines that deliver it. As the engine becomes more powerful, if the pipelines are insufficient, the entire system will struggle. HBM is the critical, albeit expensive, component in this system.
For investors and industry insiders, understanding the technical challenges and industry dynamics is crucial. Those who master the most advanced manufacturing processes and have strong partnerships with key customers will hold the upper hand in the future. This "storage capacity" cycle is just beginning.