Nvidia Corporation co-founder and CEO Jensen Huang at the Nvidia GPU Technology Conference (GTC) in San Jose, California, USA, on Tuesday, March 19, 2024.
David Paul Morris | David Paul Morris Bloomberg | Getty Images
NVIDIA A 27% gain in May brought its market capitalization to $2.7 trillion, trailing only Microsoft and apple Become one of the world’s most valuable public companies. The chipmaker reported that its sales tripled year-on-year for the third consecutive quarter as demand for artificial intelligence processors soared.
Mizuho Securities estimates that Nvidia controls 70% to 95% of the market for AI chips used to train and deploy models such as OpenAI’s GPT. The 78% gross margin highlights Nvidia’s pricing power and is a surprisingly high number for a hardware company that must make and ship physical products.
Competitor chip manufacturers Intel and Advanced Micro Devices Gross profit margins in the latest quarter were 41% and 47% respectively.
NVIDIA’s position in the AI chip market has been described by some experts as a moat. Its flagship AI graphics processing units (GPUs) like the H100, combined with the company’s CUDA software, put it so far ahead of the competition that turning to alternatives was almost unthinkable.
Still, Nvidia Chief Executive Jensen Huang, whose net worth has grown from $3 billion to about $90 billion over the past five years, said he was “concerned and worried” about the 31-year-old company losing its edge. He acknowledged at a conference late last year that many powerful competitors were emerging.
“I don’t think people are trying to bankrupt me,” Huang explain November. “I probably know they’re working on it, so that’s different.”
Nvidia has promised to release There are new AI chip architectures every yearrather than launching new software every other year as it has historically done, it can more deeply solidify its chips’ position in artificial intelligence software.
But Nvidia’s GPUs aren’t the only chips capable of running the complex mathematical operations that power generative artificial intelligence. If less powerful chips can do the same job, Huang’s paranoia may be justified.
The shift from training AI models to so-called inference (or deploying models) may also provide companies with an opportunity to replace Nvidia GPUs, especially if they are cheaper to buy and run. Nvidia’s flagship chips cost around $30,000 or more, giving customers plenty of incentive to look for alternatives.
“Nvidia wants to have 100% of the data, but customers won’t like Nvidia to have 100% of the data,” said Sid Sheth, co-founder of ambitious rival D-Matrix. “It’s such a big opportunity. It would be unhealthy for any one company to seize it all.”
Founded in 2019, D-Matrix plans to release a semiconductor card for servers later this year, aiming to reduce the cost and latency of running artificial intelligence models. company Upregulate It was $110 million in September.
In addition to D-Matrix, companies ranging from multinationals to emerging startups are vying for a share of the artificial intelligence chip market, which could reach $400 billion in annual sales over the next five years, according to market analysts and AMD. Nvidia generated about $80 billion in revenue over the past four quarters, and Bank of America estimated the company’s AI chip sales last year at $34.5 billion.
Many companies adopting Nvidia GPUs believe that different architectures or certain trade-offs can produce better chips for specific tasks. Device makers are also developing technology that could eventually do the vast amount of AI computing now done on large GPU-based clusters in the cloud.
“No one can deny that today Nvidia is the hardware you want to train and run AI models on,” said Fernando Vidal, co-founder of 3Fourteen Research. told CNBC. “But there has been incremental progress towards leveling the playing field, from very large companies developing their own chips, to smaller startups designing their own chips.”
AMD CEO Lisa Su wants investors to believe there’s enough room in the space for many successful companies.
“The key is there are a lot of options out there,” Su told reporters in December, when her company unveiled its latest artificial intelligence chip. “I think we’re going to see a situation where there’s not just one solution but multiple solutions.”
Other large chip manufacturers
Lisa Su demonstrated the AMD Instinct MI300 chip during her keynote speech at CES 2023 in Las Vegas, Nevada on January 4, 2023.
David Becker | Getty Images
AMD makes GPUs for gaming and, like Nvidia, is applying them to artificial intelligence in data centers. Its flagship chip is the Instinct MI300X. Microsoft has purchased AMD processors and provides access to them through its Azure cloud.
At the launch event, Su emphasized the chip’s superior performance in inference rather than training competition with Nvidia. Last week, Microsoft said it was using AMD Instinct GPUs to power its Copilot models. Morgan Stanley analysts believe that this news shows that AMD’s artificial intelligence chip sales this year may exceed the company’s public target of US$4 billion.
Intel, whose revenue was surpassed by Nvidia last year, is also trying to gain a foothold in the field of artificial intelligence. The company recently released Gaudi 3, the third version of its AI accelerator. Training models is faster.
Analysts at Bank of America recently estimated that Intel’s share of the artificial intelligence chip market will be less than 1% this year. Intel said it has a $2 billion backlog of orders for the chip.
The main barrier to wider adoption may be software. Both AMD and Intel are participating in a conference called “ UXL Foundationinclude Googlea company dedicated to creating a free alternative to Nvidia CUDA, the hardware used to control artificial intelligence applications.
NVIDIA’s top customers
One potential challenge for Nvidia is that it is competing with some of its largest customers. Cloud providers include Google, Microsoft and Amazon Both are building processors for internal use. The Big Three of tech giants, plus Oracleaccounting for more than 40% of Nvidia’s revenue.
Amazon launched its own artificial intelligence chip in 2018, under the brand name Inferentia. Inferentia is now available in its second edition. In 2021, Amazon Web Services debuted Traium for training. Customers can’t buy the chips, but can rent systems through AWS, which claims the chips are more cost-effective than Nvidia’s chips.
Google is probably the cloud provider most committed to its own chips. Since 2015, the company has been using so-called tensor processing units (TPUs) to train and deploy artificial intelligence models. In May, Google released the sixth version of its chip, Trillium, which the company said was used to develop its models, including Gemini and Imagen.
Google also uses Nvidia chips and delivers them through its cloud.
Microsoft hasn’t gone that far yet. company said last year The company is building its own artificial intelligence accelerators and processors, called Maia and Cobalt.
Yuan It’s not a cloud provider, but the company needs a lot of computing power to run its software and website and serve ads. Although Facebook’s parent company is buying billions of dollars worth of Nvidia processors, the company said in April that some of its domestic chips were already deployed in data centers and were “more efficient” than GPUs.
JPMorgan analysts estimated in May that the market for building customized chips for large cloud providers could be worth as much as $30 billion, with potential annual growth of 20%.
Start-up company
Cerebras’ WSE-3 chip is one example of a new class of chips from an upstart designed to run and train artificial intelligence.
brain systems inc.
Venture capitalists see an opportunity for emerging companies to join the game. They invested $6 billion in AI semiconductor companies in 2023, up slightly from $5.7 billion a year ago, according to PitchBook data.
This is a tough area for startups because semiconductors are expensive to design, develop and manufacture. But there are also opportunities for differentiation.
For Silicon Valley artificial intelligence chip maker Cerebras Systems, the focus is on the basic operations and bottlenecks of artificial intelligence, rather than the more general nature of GPUs. The company was founded in 2015 and was valued at $4 billion in its latest financing, according to Bloomberg.
The Cerebras chip WSE-2 combines GPU functionality along with central processing and additional memory into a single device, which is better suited for training large models, CEO Andrew Feldman said.
“We use one giant wafer and they use many small wafers,” Feldman said. “They have the challenge of moving data and we don’t.”
Feldman said his company, part of the Mayo Clinic GSKAnd the U.S. military is winning business as a customer for its supercomputing systems, even competing with Nvidia.
“There’s a lot of competition, and I think that’s healthy for the ecosystem,” Feldman said.
D-Matrix’s Sheth said his company plans to release a card later this year with a small chip that will allow more calculations to be done in memory rather than on a chip such as a GPU. D-Matrix’s product plugs into AI servers alongside existing GPUs, but it offloads work from Nvidia chips and helps reduce the cost of generative AI.
Customers are “very receptive and very willing to bring new solutions to market,” Sheth said.
Apple and Qualcomm
On September 22, 2023, Apple iPhone 15 series devices were launched at The Grove Apple retail store in Los Angeles, California.
Patrick T. Fallon AFP | Getty Images
The biggest threat to Nvidia’s data center business may be changes in processing locations.
Developers are increasingly betting that artificial intelligence work will move from server farms to the laptops, PCs and phones we own.
The large models developed by OpenAI require massive clusters of powerful GPUs for inference, but companies like Apple and Microsoft are developing “smaller models” that require less power and data and can run on battery-powered devices. They may not be as proficient as the latest version of ChatGPT, but they also perform other applications such as summarizing text or visual search.
Apple and Qualcomm are updating their chips to run artificial intelligence more efficiently, adding specialized parts called neural processors to AI models, which can have privacy and speed advantages.
Qualcomm recently released a PC chip that allows laptops to run Microsoft artificial intelligence services on the device. The company has also invested in several chip manufacturers to produce low-power processors to run artificial intelligence algorithms outside of smartphones or laptops.
Apple has been marketing its latest laptops and tablets as optimized for artificial intelligence because of the neural engine on the chip. At its upcoming developer conference, Apple It plans to showcase a range of new artificial intelligence features that could run on the company’s chips that power iPhones.
About The Author
