AI has become a new growth point for global technological breakthroughs, what is the key to restricting the development of AI? In addition to the research and development of the big model itself, another key is - arithmetic power. Most of the current AI training and reasoning costs are on GPU chips. In large and small AI startups around the world, training models rely on NVIDIA's H100 and other chips, a piece of an H100 chip is priced at 35,000 U.S. dollars, but the supply does not meet the demand.

Local time on February 21, NVIDIA announced the fourth fiscal quarter and 2024 fiscal year results. The results showed that NVIDIA's revenue in the quarter ended January 28 was $22.1 billion, higher than analysts' expectations of $20.41 billion, an increase of 22% from a year earlier, and a jump of 265% year-on-year. Fourth-quarter net income of $12.3 billion jumped 765 percent from a year earlier. Full-year revenue hit a record high of $60.9 billion, up 126 percent. On the eve of the earnings announcement, NVIDIA shares fell for four consecutive trading days, and the exceeded expectations of the earnings report let NVIDIA shares in the U.S. stock market after hours once rose more than 9%.

NVIDIA's soaring borrowed the east wind of the big AI model 2022 November OpenAI released ChatGPT, a chatbot trained on NVIDIA's A100 chip. The news, when revealed, triggered one of the biggest one-day gains in stock market history.

In a story, The New Yorker quoted a Wall Street analyst as saying, "There's a war going on in artificial intelligence, and NVIDIA is the only arms dealer. "

But the fiery battlefield attracts a swarm of heroes, and the major chip manufacturers and cloud service providers have to grab the new track of AI arithmetic: in April 2023, Amazon released the AI inference chip Inferentia 2; in May, Meta announced the AI chip project MITA; in August, Google released TPU v5e; in November, Microsoft released the AI chip Maia 100. in addition. In addition, there are Sun Masayoshi, Sam Altman, and other giants are eyeing the AI chip field.

NVIDIA, can still retain its monopoly in the field of AI chips, but will still be the only arms dealer?

Nvidia: Patient Monopolist

February 16, OpenAI released the Vincennes video model Sora detonated the world, and as of the close of the day, Nvidia's market value reached 1.79 trillion U.S. dollars. In 2023, Nvidia's market value rose more than twice, and this year only one and a half months, Nvidia's share price rose again by more than 50%.

Currently, NVIDIA is a monopoly in the field of AI chips. Nvidia's AI chip has how much money? The material cost of the H100 chip is 3,000 U.S. dollars, the official selling price of 35,000 U.S. dollars, the second-hand platform trading price of 50,000 U.S. dollars, the price than the cost of the overflow of 10 times, is still the market supply is not enough to meet the demand. Elon Musk said: "H100 is more difficult to buy than drugs. "

How hard currency is NVIDIA's H100? It can be used as collateral to borrow money just like a house, land, or currency. In August 2023, US startup CoreWeave got $2.3 billion in debt financing from Wall Street by pledging the H100 chip it had on hand, according to the Fargawa Institute. Not only that, the number of H100 chips in CoreWeave's hands at the time was not enough to reach $2.3 billion, but with NVIDIA's committed supply contract for H100 to CoreWeave, it was able to borrow a huge amount of money from the bank, similar to the year when real estate developers were able to borrow from the bank through the land acquisition contract and land grant.

Large and small AI startups around the world rely on NVIDIA's H100 chip to train their models. According to consulting firm Omidia, NVIDIA sold nearly 500,000 H100 chips in the third quarter of 2023, and Meta and Microsoft are the largest buyers of NVIDIA's H100 GPUs, each purchasing as many as 150,000 of them, far more than the number of H100 chips purchased by Google, Amazon, Oracle, and Tencent (50,000 each).

Monopolies aren't built in a day, and it takes time. In the early years, AI in image recognition speech recognition, and other basic tasks stagnation, the future of AI is still confusing, a lot of AI training is on NVIDIA graphics cards. 2009, known as the father of artificial intelligence Geoffrey Hinton on the use of NVIDIA's CUDA platform to train neural networks to recognize human speech. 2012, his students Alex Krizhevsky, Krizhevsky, Krizhevsky, Tencent, and other students trained neural networks to recognize human speech. Krizhevsky and Ilya Sutskever (who later became OpenAI's chief scientist) purchased two GeForce graphics cards from Amazon on a tight budget. In 2016, NVIDIA delivered the first dedicated AI supercomputer, the DGX-1, to an OpenAI research team. Jen-Hsun Huang personally brought the computer to the OpenAI's offices, and Elon Musk, then OpenAI's chairman, opened the package with a paper cutter.

OpenAI released the GPT-4 in March 2023, once again adding fuel to the fire of the entire AI industry. According to information obtained by chip researcher SemiAnalysis, OpenAI trained with 25,000 NVIDIA A100 GPUs for more than three months to make the GPT-4 big model.

But as Jen-Hsun Huang himself said, "We don't need to pretend that the company is always in danger, we are always in danger." The AI chip race has been ringed by several giants: Masayoshi Son of SoftBank Group is ready to make a big investment of $100 billion in AI chips as usual, with 70% of the funds coming from the Middle East; Microsoft is joining forces with Intel to make chips to reduce the number of chips in the market; and Microsoft has been working with Intel to make chips to reduce the number of chips in the market. Intel manufacturing chips to reduce dependence on Nvidia; more rookie teams on the underlying logic of the AI chip innovation, from Google's early TPU team, jumped out of the Groq team, with a new technology line to design chips.

Softbank's Masayoshi Sun's Investment in Chips Making

According to Bloomberg on February 15th, SoftBank Group founder Masayoshi Son is considering creating a $100 billion chip company to develop AI chips. One scenario that Masayoshi Son is considering is that SoftBank would provide $30 billion and another $70 billion could come from Middle Eastern institutions, according to people familiar with the matter. Shares of SoftBank rose 3 percent a few days after the news was disclosed.

The codename for the chip project, "Izanagi," which is Japanese for the god who opened up the world in Japanese mythology, is a sign of Masayoshi's ambition. If the Izanagi chip project is successful, it will capture about one-fifth of the global semiconductor market.

The new company is designed to be able to complement SoftBank's chip design firm Arm, people familiar with the matter said. Arm's chip design business is now one of the focuses of SoftBank Group after a series of investments by the group suffered a setback a few years ago. Arm does not make cores but designs instruction sets for modern chips, and its customers include Qualcomm, Apple, and Samsung. Softbank group holds about 90% of the shares of Arm, in February this year, Arm shares have risen 67%.

Masayoshi Sun also likes to bet big, in many historical nodes, see the trend of the times, on a wide range of bets. In 2000, when Alibaba was still a nobody, the Internet economy had not ushered in the mobile era of takeoff, Sun Masayoshi talked to Jack Ma a few minutes after Alibaba invested 20 million dollars. Because he felt that the Internet had a future, Masayoshi Son invested a large sum of money in Yahoo in 1995. Because he thought that technology had a future, he bought 80% of Kingston's shares in 1996 for 1.5 billion dollars. Of course, Masayoshi Sun's wide-ranging investments also have both gains and losses.

Sun's keen sense of the trend of the times naturally will not miss the new track of AI. According to Bloomberg, Masayoshi Son said: "AGI is the goal pursued by every artificial intelligence expert, but when you ask them for detailed definitions, numbers, time, computing power, how much smarter AGI is than human intelligence, most of them don't have an answer. I have my answer: I believe AGI will be a reality within 10 years."

It is worth mentioning that the Middle Eastern capital is a major player in the chip race and one of the most important sources of funding for Masayoshi Son. seventy percent of the funding for the Izanagi chip project came from Middle Eastern institutions. Previously, Sun founded the Vision Fund of 100 billion dollars of capital one of the world's largest pool of technology investment funds, of which 45 billion dollars from the Saudi Arabia national fund, and 15 billion dollars from the UAE national fund.

Microsoft: Do Not Rely on Others

In the new track of AI, Microsoft, Google, and Amazon have also earned a share of the pie by selling cloud services.

Microsoft is one of NVIDIA's largest customers, but Microsoft is also reducing its dependence on NVIDIA chips in various ways. In terms of developing its NICs and its chips, The Information reported on Feb. 20 that Microsoft is developing an NIC to ensure that data moves quickly between its servers as an alternative to NVIDIA-provided NICs.

Microsoft is reportedly working on its own Maia AI server chip, which will be installed in data centers this year, and the new NIC it is developing could also improve the performance of the Maia chip.

Microsoft is also joining forces with Intel to build a chip. Microsoft is designing chips that will be made in one of Intel's most advanced factories, Chief Executive Satya Nadella said at an Intel event, the Wall Street Journal reported on Feb 22nd. Nadella didn't specify which chips Intel would produce for it, but Microsoft has been looking to bolster its chip design capabilities in recent months, including a new chip for artificial intelligence computing that was introduced last year.

Previously, Microsoft had been buying AMD chips in large quantities. In December 2023, AMD launched the Instinct MI300X chip, designed as a pure GPU that delivers breakthrough performance for AI job execution. According to Seeking Alpha, Citigroup's analysts report that Microsoft's data center division is the largest buyer of AMD's Instinct MI300X chips and has already begun executing work on large-scale language models (LLMs) such as GPT-4.

Groq, a New Force on the Rise

Just two days before the release of NVIDIA's earnings report, NVIDIA suddenly appeared a formidable enemy.

On February 21, a startup called Groq detonated in AI circles. In traditional generative AI, it takes time for the Q&A to generate answers, with characters popping up one by one. On Groq's cloud service experience platform, the model can generate answers almost immediately when it receives a prompt, one second a screen, according to Tencent.

Eight of Groq's founding team came from Google's early TPU core design team of just 10 people, according to TechCrunch Daily. Although the team is off the Google TPU, Groq neither chose the TPU route nor looked at GPUs, CPUs, etc. Groq chose a completely new system route - LPU (Language Processing Unit).

"We [do] not do big models," Groq said, "Our LPU inference engine is a new end-to-end processing unit system that delivers the fastest inference speeds for compute-intensive applications such as AI big models."

Currently, Groq's LPU card is priced at around $20,000, which is much cheaper than NVIDIA's H100 GPU. The Groq team was awarded will become a new force in the field of AI chips, with a new technology path to bend the road.

These are not the only players who are planning to build AI chips.

Just last month, Sam Altman visited the South Korean chip giants Samsung and SK to explore the possibility of cooperation in making chips. Before that, he also contacted Intel and TSMC. sam Altman even proposed to use 5-7 trillion dollars to build AI chips. Many people think Sam Altman is crazy, but he saw the real bottleneck of AI development - arithmetic.

The current tenfold premium for Nvidia's AI chips comes from the huge research and development costs of the chip and software ecosystem on the one hand and from monopolization on the other.

According to domestic AI entrepreneur, Logenic AI founder Li Bojie, there is a saying in the AI circle that as long as the demand and production of chips are large enough, the chip itself is the price of sand.

Therefore, if the day when Nvidia's chips are broken from the monopoly arrives with the rapid progress of the AI track, more companies in more countries and regions will usher in the freedom of the chip, but what we need to face will be even more brand-new problems.