In 1997 the IBM Deep Blue supercomputer defeated world chess champion Garry Kasparov. It was a groundbreaking demonstration of supercomputer expertise and a primary glimpse into how high-performance computing would possibly someday overtake human–degree intelligence. Within the 10 years that adopted, we started to make use of artificial intelligence for a lot of sensible duties, akin to facial recognition, language translation, and recommending motion pictures and merchandise.
Quick-forward one other decade and a half and synthetic intelligence has superior to the purpose the place it may possibly “synthesize data.” Generative AI, akin to ChatGPT and Stable Diffusion, can compose poems, create paintings, diagnose illness, write abstract studies and computer code, and even design built-in circuits that rival these made by people.
Great alternatives lie forward for synthetic intelligence to develop into a digital assistant to all human endeavors. ChatGPT is an effective instance of how AI has democratized using high-performance computing, offering advantages to each particular person in society.
All these marvelous AI functions have been attributable to three components: improvements in environment friendly machine-learning algorithms, the supply of large quantities of knowledge on which to coach neural networks, and progress in energy-efficient computing by the development of semiconductor expertise. This final contribution to the generative AI revolution has acquired lower than its justifiable share of credit score, regardless of its ubiquity.
During the last three many years, the most important milestones in AI have been all enabled by the modern semiconductor expertise of the time and would have been not possible with out it. Deep Blue was carried out with a mixture of 0.6- and 0.35-micrometer-node chip-manufacturing expertise. The deep neural community that gained the ImageNet competitors, kicking off the present period of machine studying, was implemented with 40-nanometer technology. AlphaGo conquered the game of Go utilizing 28-nm expertise, and the preliminary model of ChatGPT was skilled on computer systems constructed with 5-nm expertise. The newest incarnation of ChatGPT is powered by servers utilizing much more superior 4-nm technology. Every layer of the pc programs concerned, from software program and algorithms right down to the structure, circuit design, and system expertise, acts as a multiplier for the efficiency of AI. However it’s honest to say that the foundational transistor-device expertise is what has enabled the development of the layers above.
If the AI revolution is to proceed at its present tempo, it’s going to wish much more from the semiconductor business. Inside a decade, it’s going to want a 1-trillion-transistor GPU—that’s, a GPU with 10 occasions as many gadgets as is typical in the present day.
Advances in semiconductor expertise [top line]—together with new supplies, advances in lithography, new forms of transistors, and superior packaging—have pushed the event of extra succesful AI programs [bottom line]
Relentless Progress in AI Mannequin Sizes
The computation and reminiscence entry required for AI coaching have elevated by orders of magnitude prior to now 5 years. Coaching GPT-3, for instance, requires the equal of greater than 5 billion billion operations per second of computation for a complete day (that’s 5,000 petaflops-days), and three trillion bytes (3 terabytes) of reminiscence capability.
Each the computing energy and the reminiscence entry wanted for brand spanking new generative AI functions proceed to develop quickly. We now must reply a urgent query: How can semiconductor expertise preserve tempo?
From Built-in Units to Built-in Chiplets
Because the invention of the built-in circuit, semiconductor expertise has been about cutting down in characteristic dimension in order that we are able to cram extra transistors right into a thumbnail-size chip. In the present day, integration has risen one degree greater; we’re going past 2D scaling into 3D system integration. We are actually placing collectively many chips right into a tightly built-in, massively interconnected system. It is a paradigm shift in semiconductor-technology integration.
Within the period of AI, the potential of a system is immediately proportional to the number of transistors integrated into that system. One of many important limitations is that lithographic chipmaking instruments have been designed to make ICs of not more than about 800 sq. millimeters, what’s referred to as the reticle restrict. However we are able to now prolong the scale of the built-in system past lithography’s reticle restrict. By attaching a number of chips onto a bigger interposer—a chunk of silicon into which interconnects are constructed—we are able to combine a system that comprises a a lot bigger variety of gadgets than what is feasible on a single chip. For instance, TSMC’s chip-on-wafer-on-substrate (CoWoS) expertise can accommodate as much as six reticle fields’ value of compute chips, together with a dozen high-bandwidth-memory (HBM) chips.
HBMs are an instance of the opposite key semiconductor expertise that’s more and more necessary for AI: the power to combine programs by stacking chips atop each other, what we at TSMC name system-on-integrated-chips (SoIC). An HBM consists of a stack of vertically interconnected chips of DRAM atop a management logic IC. It makes use of vertical interconnects referred to as through-silicon-vias (TSVs) to get indicators by every chip and solder bumps to type the connections between the reminiscence chips. In the present day, high-performance GPUs use HBMextensively.
Going ahead, 3D SoIC expertise can present a “bumpless different” to the traditional HBM expertise of in the present day, delivering far denser vertical interconnection between the stacked chips. Current advances have proven HBM test structures with 12 layers of chips stacked utilizing hybrid bonding, a copper-to-copper reference to a better density than solder bumps can present. Bonded at low temperature on high of a bigger base logic chip, this reminiscence system has a complete thickness of simply 600 µm.
With a high-performance computing system composed of numerous dies operating giant AI fashions, high-speed wired communication could shortly restrict the computation pace. In the present day, optical interconnects are already getting used to attach server racks in knowledge facilities. We are going to quickly want optical interfaces primarily based on silicon photonics that are packaged together with GPUs and CPUs. It will permit the scaling up of energy- and area-efficient bandwidths for direct, optical GPU-to-GPU communication, such that a whole bunch of servers can behave as a single large GPU with a unified reminiscence. Due to the demand from AI functions, silicon photonics will develop into one of many semiconductor business’s most necessary enabling applied sciences.
Towards a Trillion Transistor GPU
As famous already, typical GPU chips used for AI coaching have already reached the reticle discipline restrict. And their transistor rely is about 100 billion gadgets. The continuation of the pattern of accelerating transistor rely would require a number of chips, interconnected with 2.5D or 3D integration, to carry out the computation. The mixing of a number of chips, both by CoWoS or SoIC and associated superior packaging applied sciences, permits for a a lot bigger whole transistor rely per system than may be squeezed right into a single chip. We forecast that inside a decade a multichiplet GPU may have greater than 1 trillion transistors.
We’ll must hyperlink all these chiplets collectively in a 3D stack, however happily, business has been in a position to quickly scale down the pitch of vertical interconnects, rising the density of connections. And there’s loads of room for extra. We see no purpose why the interconnect density can’t develop by an order of magnitude, and even past.
Vitality-Environment friendly Efficiency Pattern for GPUs
So, how do all these progressive {hardware} applied sciences contribute to the efficiency of a system?
We are able to see the pattern already in server GPUs if we take a look at the regular enchancment in a metric referred to as energy-efficient efficiency. EEP is a mixed measure of the vitality effectivity and pace of a system. Over the previous 15 years, the semiconductor business has elevated energy-efficient efficiency about threefold each two years. We imagine this pattern will proceed at historic charges. It is going to be pushed by improvements from many sources, together with new supplies, system and integration expertise, extreme ultraviolet (EUV) lithography, circuit design, system structure design, and the co-optimization of all these expertise components, amongst different issues.
Specifically, the EEP enhance will likely be enabled by the superior packaging applied sciences we’ve been discussing right here. Moreover, ideas akin to system-technology co-optimization (STCO), the place the completely different purposeful components of a GPU are separated onto their very own chiplets and constructed utilizing the very best performing and most economical applied sciences for every, will develop into more and more vital.
A Mead-Conway Second for 3D Built-in Circuits
In 1978, Carver Mead, a professor on the California Institute of Expertise, and Lynn Conway at Xerox PARC invented a computer-aided design method for integrated circuits. They used a set of design guidelines to explain chip scaling in order that engineers may simply design very-large-scale integration (VLSI) circuits with out a lot data of course of expertise.
That very same kind of functionality is required for 3D chip design. In the present day, designers must know chip design, system-architecture design, and {hardware} and software program optimization. Producers must know chip expertise, 3D IC expertise, and superior packaging expertise. As we did in 1978, we once more want a standard language to explain these applied sciences in a method that digital design instruments perceive. Such a {hardware} description language provides designers a free hand to work on a 3D IC system design, whatever the underlying expertise. It’s on the best way: An open-source normal, referred to as 3Dblox, has already been embraced by most of in the present day’s expertise firms and digital design automation (EDA) firms.
The Future Past the Tunnel
Within the period of synthetic intelligence, semiconductor expertise is a key enabler for brand spanking new AI capabilities and functions. A brand new GPU is not restricted by the usual sizes and type components of the previous. New semiconductor expertise is not restricted to cutting down the next-generation transistors on a two-dimensional aircraft. An built-in AI system may be composed of as many energy-efficient transistors as is sensible, an environment friendly system structure for specialised compute workloads, and an optimized relationship between software program and {hardware}.
For the previous 50 years, semiconductor-technology improvement has felt like strolling inside a tunnel. The street forward was clear, as there was a well-defined path. And everybody knew what wanted to be executed: shrink the transistor.
Now, now we have reached the top of the tunnel. From right here, semiconductor expertise will get more durable to develop. But, past the tunnel, many extra prospects lie forward. We’re not sure by the confines of the previous.