Huang Renxun: Every day feels like the first day of entrepreneurship! I sacrificed everything for NVIDIA.

Compiled from Tencent Technology. Key Points ① Huang Renxun admits that every day he wakes up, it feels like the first day of entrepreneurship because NVIDIA is always exploring unknown territories with the possibility of failure. ② Huang Renxun revealed that NVIDIA is working hard to transform the so-called "AI factory" into actual products, a new form of data center. ③ Huang Renxun stated that NVIDIA is making breakthrough progress in basic robot technology, and the arrival of humanoid robots may be just around the corner. ④ When asked whether the US chip export restrictions would stimulate China to launch more competitive AI chips, Huang Renxun mentioned that China has many highly competitive assets. According to the US tech magazine "Wired," the conversation with Huang Renxun, co-founder and CEO of NVIDIA, felt like an adventurous journey in the field of technology. Huang Renxun's strong belief and unique insights in artificial intelligence left Wired journalist Lauren Goode full of anticipation for the future after a lively discussion lasting nearly 90 minutes. Huang Renxun firmly believes that neural networks will dominate the future, and fields such as robotics, healthcare, and autonomous driving cars will see tremendous development opportunities. He even foresees that chatbots with memory will become a reality. The architectural style of NVIDIA in the Santa Clara area also reflects its unique spirit of innovation. Everywhere you go, you can see triangles nested within triangles, a design element that is not only aesthetically pleasing but also once helped NVIDIA earn its first fortune. This spirit deeply attracted Goode, who was captivated by Huang Renxun's vision. As a prominent figure in the current tech scene, Huang Renxun's influence is undeniable. This year, and even in the next decade, he will remain a significant figure in the tech industry. The demand for NVIDIA's supercomputing graphics processing units (GPUs) seems endless, thanks to NVIDIA's outstanding performance in graphics processing and its keen insight into future technology trends. During the interview, Huang Renxun wore his iconic leather jacket and black sneakers. He admitted that he dislikes Monday mornings because working all day on Sunday leaves him exhausted. However, just two days later, at a medical investment seminar, he stood on stage again, energetically delivering a speech to the attendees. In his speech, Huang Renxun expressed his views and expectations for future technology. He mentioned that biologists and scientists are passionate groups who use terms like "target" and "inhibition" to describe their research, while NVIDIA hopes to drive technological development using words like "creation," "improvement," and "acceleration." He emphasized, "If you want to complete drug design and discovery on silicon chips (using AI to help design and discover brand-new drugs), handling large amounts of data will be essential. For those facing challenges in AI computing, they can turn to NVIDIA for help." 黄仁勋's success is not by chance. With his visionary perspective and strategic thinking, he always manages to accurately grasp the forefront of every major technological trend. From the birth of the AlexNet image recognition system in 2012 to the rise of transformer neural network architecture in 2017, NVIDIA has always been at the forefront of artificial intelligence technology. This has also enabled NVIDIA to dominate the artificial intelligence chip market with an absolute leading position. Currently, NVIDIA holds over 80% of the artificial intelligence chip market sales, with a market value close to $2 trillion. The company's revenue in the last quarter of 2023 reached $22 billion, a 265% increase year-on-year. Its stock price rose by 231% last year. Huang Renxun's success is remarkable, whether it's due to exceptional talent and profound knowledge of technology, extraordinary luck in seizing the pulse of the times, or a combination of both talent and timing! Everyone is curious, how did he achieve this? However, competition in the technology industry is fierce. Huang Renxun is well aware that he is in the midst of the chip war and also under the scrutiny of regulatory agencies. His challengers include well-known brands such as Google, Amazon, Meta, and Microsoft, among other tech giants. In addition, startup companies are also eyeing NVIDIA's market share. According to data from the research firm Pitchbook, in just the third quarter of last year, venture capitalists invested over $800 million in the artificial intelligence chip sector. Nevertheless, Huang Renxun has never stopped moving forward. Even during interviews, he is contemplating how to better address future challenges. He questions where Good comes from and how he lives in the Bay Area. Their conversation unfolds from there: The following is the full interview with Huang Renxun: Huang Renxun: We are both graduates of Stanford University. Lauren Good: Yes. I studied journalism, but you didn't. Huang Renxun: I really wish I had studied journalism. Lauren Good: Why do you say that? Huang Renxun: One person I greatly admire, whether as a leader or an individual, is Shantanu Narayen, the CEO of Adobe. He said he always wanted to be a journalist because he loves telling stories. Lauren Good: Effectively telling your own story seems to be an important part of entrepreneurship. Huang Renxun: Yes. Setting strategies is telling stories, and building corporate culture is also telling stories. Lauren Good: You have mentioned many times that you don't pitch NVIDIA's ideas through a Pitch Deck. Huang Renxun: That's right, I really want to tell a good story. Lauren Good: So I want to start with a story told to me by another tech executive. He pointed out that NVIDIA was founded a year earlier than Amazon, but in many ways, NVIDIA has a stronger "entrepreneurial first day" mentality than Amazon. How did you achieve this? 黄仁勋: To be honest, this is a very apt description. Every morning when I wake up, it feels like the first day of entrepreneurship because we are always exploring unknown territories and trying things we have never done before. However, this also means we will face risks and the possibility of failure. Just now, I attended a meeting where we were discussing a completely new project for our company. We are clueless about it and unsure how to successfully implement it. 劳伦·古德: Anything new? 黄仁勋: We are building an advanced artificial intelligence factory, a new form of data center. Unlike traditional data centers where multiple users share a computer cluster and store files, the AI factory is more like an independent power generator. After years of meticulous research and construction, we have completed the prototype of the AI factory. Now, we are challenged with transforming it into an actual product. 劳伦·古德: What do you plan to name it? 黄仁勋: This yet-to-be-named innovative project is destined to be ubiquitous. Both cloud service providers and ourselves will be dedicated to building it. Biotech companies, retail companies, logistics companies, and even future car companies will have it. It will not only be a factory for manufacturing cars but also for manufacturing artificial intelligence for cars. In fact, as we speak, Elon Musk is already at the forefront of this trend. His profound insights into the future of industry and forward-thinking have left most people in awe. 劳伦·古德: You have mentioned before that you manage a flat organizational structure where 30 to 40 executives report directly to you, allowing you to be deeply involved in the flow of information. So, what recent events or perspectives have sparked your enthusiasm and led you to the idea of going all-in on NVIDIA's future? 黄仁勋: In the era of cavemen, information transmission did not rely on modern communication methods like emails and text messages but flowed freely within communities. Today, with the rapid advancement of technology, the speed of information flow far exceeds what it used to be, rendering traditional hierarchical information structures inadequate for current needs. A flat organizational structure, with its efficient information transmission and decision-making speed, enables us to quickly adapt to this ever-changing world. Taking NVIDIA's technological development as an example, the iteration speed of Moore's Law has witnessed exponential growth in technology. In just the past decade, the capabilities of artificial intelligence have increased by approximately a million times, far surpassing Moore's Law predictions. Therefore, in such a fast-paced era, information flow must be equally efficient and rapid to ensure that every level of employees can promptly access and respond to new information. 劳伦·古德: However, I am eager to delve deeper into your grand vision of the Roman Empire. What exciting visions are depicted in the current Transformer thesis (Transformer, the T in ChatGPT, an underlying algorithm architecture for artificial intelligence learning)? Do you believe that the historic changes happening now might overturn all our familiar perceptions and expectations? Huang Renxun: There are several matters worth discussing. One of them, yet to be officially named, represents a breakthrough in the field of basic robot technology that we have achieved. If machines can generate text and draw images, could they also generate actions? The answer seems to be affirmative. Once machines grasp the way to generate actions, they may be able to perceive the intentions behind them and create a universal form of expression. In this way, the arrival of humanoid robots may be just around the corner. I firmly believe that research on State Space Models (SSM) will enable us to learn extremely complex patterns and sequences without bearing the burden of quadratic growth in the computational process. This may become the prototype of the next generation of transformers. Lauren Good: What can this bring? Can you give a real example? Huang Renxun: You can have a long conversation with a computer, and the context will never be forgotten. You can even temporarily change the subject, then return to the previous topic, and retain the background of that context. You may be able to understand a very long chain of sequences, such as the human genome. Just take a look at the genetic code, and you can understand its meaning. Lauren Good: How far are we from such a future? Huang Renxun: From the birth of AlexNet to the rise of its outstanding version, the superhuman AlexNet, only five years have passed. Today, the basic models of robots are quietly emerging, heralding an upcoming technological revolution. I expect that perhaps at some point next year, we will launch it. And five years later, you will witness a series of astonishing miracles. Lauren Good: Which industry will benefit the most from widely trained robot behavior models? Huang Renxun: Heavy industry undoubtedly occupies a core position in the global industrial field. Although in modern technology, mobile electronics have become relatively easy, compared to this, moving atoms appears extremely challenging. Transportation and logistics, as vital pillars of the global economy, involve efficiently and safely transporting heavy objects from one place to another, requiring a deep understanding of microelements such as atoms, molecules, and proteins. These are large industries that artificial intelligence has not yet impacted. Lauren Good: You mentioned Moore's Law, is it now irrelevant? Huang Renxun: Moore's Law has evolved into a systemic issue covering a wide range of fields, not just limited to the chip industry. It focuses more on the interconnectivity and collaborative capabilities between multiple chips. About 10 to 15 years ago, we embarked on the journey of computer decomposition, enabling multiple chips to seamlessly connect and work together. Lauren Good: Is this the reason for NVIDIA's acquisition of the Israeli company Mellanox in 2019? NVIDIA stated at the time that modern computing demands a huge need for data centers, and Mellanox's network technology will make accelerated computing more efficient. Huang Renxun: Exactly. We acquired Mellanox to expand our chips, turning the entire data center into a super chip, making modern artificial intelligence supercomputers possible. In recent years, people have begun to realize the limitations of Moore's Law and understand that in order to further expand computing capabilities, it is necessary to operate on a larger scale of data centers. We have delved into the formation mechanism of Moore's Law and found that it does not actually limit the development of computers. Therefore, we must break free from the constraints of Moore's Law and think from a completely new perspective on how to achieve the continuous expansion of computing capabilities. Lauren Good: The acquisition of Mellanox by NVIDIA is now considered a very wise move. Recently, you attempted to acquire Arm, the world's most important chip intellectual property company, but faced regulatory obstacles. When considering acquisitions now, what specific aspects will you take into account? Jensen Huang: Indeed, the design of operating systems for large-scale systems faces unprecedented complexity, especially when it comes to coordinating millions, billions, or even tens of billions of tiny processors. We are very willing to collaborate with any research team in this field to advance operating system technology together. At the same time, we will continue to increase research and development investment to explore more possible solutions. Lauren Good: You have said that for NVIDIA, owning an operating system and turning it into a platform is crucial. Jensen Huang: We are a platform company. Lauren Good: Becoming a comprehensive platform does mean taking on more responsibility, especially in critical areas such as autonomous driving cars, medical devices, and artificial intelligence systems. How do you address issues such as the performance of autonomous driving cars, the margin of error in medical devices, and the potential biases in artificial intelligence systems? Jensen Huang: We are not just an application company, but dedicated to deeply serving a specific industry. In the healthcare sector, although drug development is not our expertise, we have outstanding professional capabilities in computer technology. Similarly, although we do not directly manufacture cars, we provide advanced computing solutions for the automotive industry, giving vehicles a significant advantage in artificial intelligence performance. Admittedly, it is almost impossible for a company to excel in all areas, but we firmly believe that we can be the best in the field of artificial intelligence computing. Lauren Good: There were reports last year that some customers had to wait for several months to receive your artificial intelligence GPUs. How is the situation now? Jensen Huang: I believe our supply will continue to be constrained this year. We may not be able to meet demand this year and even next year. Lauren Good: How long is the current waiting time? Jensen Huang: I am not sure about the current delivery time. However, you know, this year marks the beginning of our supply of the new generation processors. Lauren Good: Are you referring to Blackwell, the rumored new graphics processor? Jensen Huang: Yes, the new generation of GPUs has arrived, and Blackwell's performance has set a new record. It will be incredible. Lauren Good: Does this mean that customers will need fewer GPUs? 黄仁勋: This is our goal. We aim to significantly reduce the cost of training large models, allowing people to scale up the models they want to train. Lauren Good: NVIDIA has invested in many AI startups. There were reports last year that you invested in over 30 companies. Have those startups been squeezed out of the queue while waiting for your hardware? Huang Renxun: They are facing the same supply challenges as many other companies. Since most companies rely on public cloud services, they have to negotiate directly with public cloud service providers. However, they can leverage our AI technology, which means they can use our engineering capabilities and special technology to optimize their AI models. We are committed to helping them improve efficiency. If their throughput increases fivefold, it essentially means they gain the computational power of five additional GPUs. This is the practical benefit they get by choosing us. Lauren Good: Do you see yourself as a kingmaker in this regard? Huang Renxun: No. We choose to invest in these companies because of the remarkable work they are doing. For us, being able to provide financial support to them is an honor, rather than them relying on us. These companies gather the top talents globally, and they do not need to enhance their reputation by using our name. Lauren Good: What will happen when machine learning shifts more towards reasoning rather than training? If AI work becomes less computationally intensive, will this reduce the demand for GPUs? Huang Renxun: We are enthusiastic about expanding the inference business. If I have to make a rough estimate, I firmly believe that NVIDIA's current business focus is already 70% on inference, with the remaining 30% focused on training. This shift is undoubtedly a cause for celebration as it signifies a significant leap in AI technology. If NVIDIA's business ratio still had 90% training dominating and only 10% inference, we can assert that AI is still in the research stage. This was the situation in the AI field seven or eight years ago. However, today, when we input a command in the cloud, it can quickly generate various content - whether it's videos, images, 2D, 3D graphics, or text, all of this often hides the powerful support of NVIDIA GPUs. Lauren Good: Do you think there will be a time when the market demand for your AI GPUs will decrease? Huang Renxun: I firmly believe that we are at the beginning of the generative AI revolution. Looking at the world today, most computing tasks still rely on traditional retrieval methods. Retrieval means that when you touch the screen of your phone, it sends a signal to the cloud to search for information. The cloud may use various technologies, such as Java, to integrate these fragmented pieces of information into a response, which eventually appears on your phone screen. However, future computing will increasingly rely on RAG - Retrieval-Augmented Generation. This is a revolutionary framework that gives large language models the ability to extract data beyond their regular parameters. In this new computing paradigm, the retrieval part will gradually decrease, while personalization and generation will take the lead. These tasks will undoubtedly be completed by efficient GPUs distributed globally. Therefore, I believe we are at the beginning of a revolution in retrieval enhancement and generative computing, where generative artificial intelligence will become an indispensable part of almost everything. Lauren Good: The latest news is that you have been cooperating with the U.S. government to develop chips that comply with export restrictions and ship them to China. My understanding is that these are not the most advanced chips. How close is your cooperation with the government to ensure that you can continue doing business in China? Huang Renxun: The U.S. has identified that NVIDIA's technology and this artificial intelligence computing infrastructure are strategically important to the country and will be subject to export controls. We initially complied with export controls in August 2022. The U.S. added more export control clauses in 2023, which led us to redesign our products. We are developing a new set of products that comply with today's export control regulations. We are working closely with the government to ensure that our proposals align with their ideas. Lauren Good: Are you concerned that these restrictions will stimulate China to launch more competitive artificial intelligence chips? Huang Renxun: China has many competitive strengths. Lauren Good: Indeed. Huawei's Mate 60 smartphone launched last year has attracted widespread attention due to its self-developed 7-nanometer chip. Huang Renxun: Huawei is a very good company. Despite the limitations of existing semiconductor processing technology, they can still build very powerful systems by aggregating many chips together. Lauren Good: Overall, are you worried that China can compete with the United States in the field of generative artificial intelligence? Huang Renxun: The implementation of these regulations will undoubtedly restrict China's ability to access cutting-edge technology. This means that in the race for technology, Western countries not subject to export controls may gain more advantages, accelerating their progress in the field of technology. For China, this undoubtedly increases the cost and difficulty of acquiring technology. Technically, China may be able to compensate for this deficiency by aggregating more chip manufacturing systems, but such a solution will undoubtedly increase unit costs and production complexity. Lauren Good: You are producing compliant chips to continue sales in China. Does this affect your relationship with TSMC? Huang Renxun: No. The regulations are specific, similar to speed limits on highways. Lauren Good: You have mentioned many times that out of the 35,000 components in your supercomputer, 8 come from TSMC. When I heard this, I thought it must be a very small part. Are you downplaying your reliance on TSMC? Huang Renxun: No, not at all! What I want to emphasize is that building an artificial intelligence supercomputer is a comprehensive project involving the integration of numerous components and technologies. In fact, in the artificial intelligence supercomputer project we are dedicated to building, almost the entire semiconductor industry is collaborating with us. We have established solid partnerships with industry leaders such as Samsung, SK Hynix, Intel, AMD, Broadcom, Marvell, and others. This collaborative model not only promotes resource sharing and technological exchange but also lays a solid foundation for our joint success. When our artificial intelligence supercomputer achieves breakthroughs and successes, it also means that this whole bunch of companies closely cooperating with us will also achieve tremendous success together. We are very pleased about this. Lauren Good: How often do you talk to Morris Chang (Founder) or C.C. Wei (CEO) of TSMC? Jensen Huang: We always keep in touch and have never interrupted. Lauren Good: What are your conversations about? Jensen Huang: These days, we have delved into advanced packaging technologies and looked ahead to the development trends in storage capacity and computing power in the coming years. Among them, CoWoS technology - TSMC's innovative method of integrating chips and memory modules into a single package has become our focus. However, to achieve the large-scale production of this technology, we need new factories, production lines, and corresponding advanced equipment. Therefore, the support of our partners is crucial. Lauren Good: I recently had a conversation with the CEO of a company focusing on developing generative artificial intelligence technology. When I asked about NVIDIA's future competitors, he mentioned Google's TPU. Others also mentioned AMD. I know it's not that simple for you, but who do you consider your biggest competitor? Who keeps you up at night? Jensen Huang: Nowadays, the entire tech industry is racing to develop and optimize their chip technologies, whether it's the TPU team, AWS Trainium and Interentia teams, Microsoft's Maia project, or major cloud service providers and startups in China, all are investing a lot of effort in this field. This competitive situation is indeed very intense, but for me, it doesn't keep me up at night. I am well aware that as long as our team works hard, remains efficient and innovative, then no matter how fierce the external competition is, we have the ability to cope. This is what I can control and what I always believe in. However, what motivates me to wake up every morning with energy is that we must continue to fulfill our commitment. That is, we must be the only company in the world that can attract everyone to collaborate and jointly build data center-scale and full-stack artificial intelligence supercomputers. Lauren Good: I have a few personal questions for you. I once asked ChatGPT a question about you. I want to know if you have any tattoos because I plan to give you a tattoo next time we meet. Jensen Huang: If you get a tattoo, I'll get one too. Lauren Good: I already have one, but I've always wanted more. Jensen Huang: I have one too. Lauren Good: I've learned from ChatGPT. According to it, when NVIDIA's stock price reached $100, Jensen Huang got the company's logo tattooed. Then it wrote: "However, Jensen Huang said he is unlikely to get another tattoo and pointed out that the pain of tattooing was more severe than he had imagined." It said you cried at that time. Did you really cry? Jensen Huang: It was a bit painful. My advice is that you should have a whiskey or take painkillers before getting a tattoo. I also believe that women can endure more pain because my daughter has a quite large tattoo. Lauren Good: If you want to get a tattoo, I think a triangle might be a good choice. Who doesn't like triangles? They are perfect geometric shapes. Huang Renxun: Or maybe a silhouette of the NVIDIA headquarters! It's also made up of triangles. Lauren Good: I'm curious, how often do you personally use tools like ChatGPT or Bard? Huang Renxun: I've been using Perplexity, and I also like ChatGPT. I use them almost every day. Lauren Good: What do you use them for? Huang Renxun: Research. For example, computer-aided drug discovery. Maybe you want to know the latest developments in computer-aided drug discovery. So you want to build the whole theme so that you have a framework. From this framework, you can ask more specific questions. I really like these large language models. Lauren Good: I heard you used to practice weightlifting before, are you still keeping up with it? Huang Renxun: No. I try to do 40 push-ups a day, which only takes a few minutes. I'm a lazy exerciser. I do squats while brushing my teeth. Lauren Good: Your comments on the Acquired podcast recently sparked a lot of discussion. The host asked you: If you were 30 years old now and thinking of starting a company, what would you do? You said you wouldn't start NVIDIA at all. Do you have any new thoughts on this? Huang Renxun: This question can be answered in two ways. And my answer is: If I knew everything I know now back then, I might have been too afraid to do it. Lauren Good: You have to be a bit delusional to start a business. Huang Renxun: That's the benefit of ignorance. You don't know how hard the future will be, you don't know how much pain and torment there will be. Nowadays, when I meet entrepreneurs and they tell me how easy entrepreneurship is, I really support them. I'm not trying to break their illusions. But deep down, I know, "Oh my, it's not going to be as easy as they imagine." Lauren Good: What do you think is the biggest sacrifice you've made in running NVIDIA? Huang Renxun: Success often comes with countless sacrifices and efforts. For entrepreneurs, this path is filled with challenges and hardships. Long hours of hard work, facing doubts and denials from the outside, lack of security, emotional vulnerability, and sometimes even humiliation are all realities that entrepreneurs must face. CEOs, entrepreneurs, and others are all human. It's embarrassing when they fail in public. So when someone says, "Lao Huang, you already have so much today, you won't start over again." But if I knew back then that NVIDIA would become what it is today, would I still have started this company? Are you kidding me? I sacrificed almost everything for this!