Nvidia's Huang rips Anthropic's proposal for AI safety antitrust waiver: 'Completely unnecessary'

Nvidia CEO Jensen Huang on Tuesday slammed proposals of artificial intelligence companies antitrust exemptions that could coordinate a slowdown of developing increasingly powerful models.

Huang's comments — in an interview with CNBC's Jim Cramer — come during an intense debate over AI safety fueled by an essay Anthropic CEO Dario Amodei published Saturday. Amodei proposed that leading AI labs coordinate to "pace the frontier," or deliberately slow the development of advanced models to give companies more time to test and safeguard. Amodei said that effort could require government mediation or waivers of antitrust restrictions.

Huang agreed that AI companies need to take safety seriously, but argued they should be able to solve those challenges themselves.

"The fact that we need new laws, new antitrust laws, or new regulations, so that these companies could do their fundamental engineering and do it properly before they release products, that is just completely unnecessary," Huang said on CNBC's "Mad Money." "We have plenty of laws. We have plenty of regulations that govern the reliability and the functionality of products."

In his essay, Amodei laid out a three-step approach to moderate the pace of model development, beginning with embedding third-party safety evaluators at each lab. The second and third steps are democratic and global coordination on safety standards and "limits on the rate of unchecked AI progress," Amodei wrote. "Some forms of coordination that would be impactful for pacing are legally challenging, and will require government support."

The legal issues arise because leading AI labs are competitors and their models are competing products. The Sherman Antitrust Act generally prohibits agreements among rivals that unreasonably restrain competition, while certain forms of coordination — including agreements to restrict output — can be illegal. That creates potential legal risks if competing AI developers were to jointly agree on how quickly they would develop or release new models.

Huang said model developers have the power to ensure their products are safe before releasing them.

"Safety is an engineering problem. Testing is an engineering problem," said Huang, whose company counts Anthropic and rival lab OpenAI as important customers. Nvidia also develops its own Nemotron family of open-weight models. "We should create products and properly test them. And if they're not ready to be released, just hold on to it and keep testing it and keep engineering until it's ready."

The AI boom has turned Nvidia into the world's largest chipmaker and most valuable company. Efforts to slow the development of more capable models may pose a risk to Nvidia's future growth.

Anthropic did not respond to CNBC's request for comment on Huang's remarks.

"AI safety is a real thing," Huang told Cramer. "Engineering products so that they are safe for the world to use — that's a real thing. And companies should innovate as fast as possible, but they should never innovate so fast as to release unsafe products."

Huang also pushed back against warnings that increasingly powerful AI models will pose an existential threat to humanity.

"We're not going to die in 2030," he said. "There are so many people in the world who are going to build AI properly. We're going to have all kinds of guardrails, invent all kinds of technology for safety and security."

Sign up now for the CNBC Investing Club to follow Jim Cramer's every move in the market.

Disclaimer