Author: Zhang Yifan
Editor: Shen Siqi
Source: Hard AI
Chip packaging has been upgraded again!
This time, the processor (CPU, GPU) and memory (DRAM) are vertically stacked.
Currently, the latest packaging technology in the world can only support horizontal stacking of different chips, also known as 2.5D packaging.
However, according to the 3D packaging technology recently released by TSMC at a forum in North America, with the entry into 3D packaging, different chips can be vertically stacked, thereby further reducing the overall size of the product — &#34;Hybrid Bonding&#34; is the key technology to achieve vertical stacking.
It is worth noting that this technology is still in the development stage, and Citigroup predicts that it will explode in 2027. As a trend for advanced packaging in the future, the new &#34;Hybrid Bonding&#34; technology in 3D packaging will bring new opportunities for semiconductor equipment manufacturers (bonding machine manufacturers, etc.), storage suppliers, and wafer foundries.
<h2>I. Advanced Packaging Enters the 3D Era from 2.5D</h2>
1) What is &#34;Hybrid Bonding&#34;?
Hybrid Bonding is a technology that connects different chips to each other.
This technology combines metal bonding and dielectric bonding, allowing direct connection of wafers or chips without the use of traditional solder bumps, reducing the chip spacing from the traditional 100 microns to 5 microns. At the same time, it achieves vertical stacking of different chips.
2) Process Flow of &#34;Hybrid Bonding&#34;
The process flow of &#34;Hybrid Bonding&#34; mainly consists of three steps: polishing the chip contact surface, aligning the wafers, and bonding with heating.
From the process perspective, this technology will increase the following requirements:
<blockquote>
• Increased use of Through-Silicon Via (TSV) technology;
• More widespread adoption of Chemical Mechanical Polishing (CMP) and etching processes;
• Increased demand for cutting, bonding, inspection, and testing processes;
</blockquote>
3) Beneficiary Links in the Industry Chain Driven by &#34;Hybrid Bonding&#34;
Due to the increased processes, semiconductor equipment manufacturers, storage suppliers, and wafer foundries will encounter new opportunities:
<blockquote>
• Equipment suppliers: BESI, ASMPT, Applied Materials;
• Memory suppliers: SK Hynix, Samsung Electronics;
• Wafer foundry supplier: TSMC;
</blockquote>
<h2>II. Power Demand Brought by AI Data Centers</h2>
There have been many reports on the surge in power demand from AI data centers, and this time Goldman Sachs has provided new forecasts.
Taking the United States as an example, the report predicts that by 2030, the power demand of data centers is expected to more than double, adding 47 GW of power demand, growing at a compound annual growth rate (CAGR) of 15%.
By 2030, data centers will account for 8% of the total power demand in the United States, higher than the current 3%. It is estimated that by 2030, the United States will need to add approximately 47GW of power capacity.
The significant growth in power demand implies huge investments in power production and transmission infrastructure, bringing opportunities to companies across the entire power supply chain, including infrastructure contractors and industrial companies manufacturing power generation and distribution products, as well as utility companies.
It is worth noting that with the continuous iteration of chip technology, the power demand for training a single model is rapidly decreasing.
At the GTC conference in March this year, NVIDIA CEO Jensen Huang revealed that the power required to train a GPT-4 model (1.8 trillion parameters) with GB200 has decreased from 15 megawatts to 4 megawatts, a three-quarters reduction in energy consumption.
Therefore, with the continuous advancement of chip technology, there is a possibility of a slowdown in the growth rate of power demand.
<h2>III. AI Server Volume Increase + AI PCs Going Public, Better Momentum in the Tech Industry in the Second Half of the Year</h2>
Recent data released by major technology consulting firms and leading companies in the AI industry chain show that AI servers will continue to increase in volume in 2024, and this year will also be the first year of volume for AI PCs, especially since Microsoft announced a new AI PC based on ARM architecture yesterday.
1) AI Servers
According to recent data released by major consulting firms and leading companies in the AI supply chain, AI servers will continue to increase in volume in the second half of the year:
• AI server supplier Foxconn expects double-digit growth in 2024;
• Quanta Management predicts that by 2024, AI server sales will account for 40% of total server sales;
2) AI PCs
Microsoft's Microsoft Build conference will start at midnight on the 22nd Beijing time and will last for 3 days
According to the company's disclosed meeting agenda, AI PC will be one of the key focuses of the conference, and it is expected that this conference will once again push AI PC into the spotlight.
Sigmaintell, a consulting firm focusing on the global high-tech industry, predicts that 2024 will be the first year of AI PC shipments, with an estimated shipment volume of around 13 million units.
It is expected that in the second half of 2024, as the AI PC market experiences rapid growth and expansion, enterprises in its industry chain will benefit

Chips are getting denser, and Citigroup believes that "hybrid bonding" technology is key

Citi reiterates 3D packaging, this time focusing on this new area | AI dehydration