"If you're from Microsoft, and you want to scale that, at the scale of Bing, that's maybe $4 billion. It's easy to see how the cost of A100s can add up.įor example, an estimate from New Street Research found that the OpenAI-based ChatGPT model inside Bing's search could require 8 GPUs to deliver a response to a question in less than one second.Īt that rate, Microsoft would need over 20,000 8-GPU servers just to deploy the model in Bing to everyone, suggesting Microsoft's feature could cost $4 billion in infrastructure spending. On Wednesday, Nvidia said it would sell cloud access to DGX systems directly, which will likely reduce the entry cost for tinkerers and researchers. This system, Nvidia's DGX A100, has a suggested price of nearly $200,000, although it comes with the chips needed. In addition to a single A100 on a card that can be slotted into an existing server, many data centers use a system that includes eight A100 GPUs working together. This means companies that find themselves with a hit AI product often need to acquire more GPUs to handle peak periods or improve their models. More computers neededĬompared to other kinds of software, like serving a webpage, which uses processing power occasionally in bursts for microseconds, machine learning tasks can take up the whole computer's processing power, sometimes for hours or days. Hopper is the code name for the new generation, including H100, which recently started shipping. "There's no question that whatever our views are of this year as we enter the year has been fairly dramatically changed as a result of the last 60, 90 days."Īmpere is Nvidia's code name for the A100 generation of chips. "The activity around the AI infrastructure that we built, and the activity around inferencing using Hopper and Ampere to influence large language models has just gone through the roof in the last 60 days," Huang said. Nvidia CEO Jensen Huang couldn't stop talking about AI on a call with analysts on Wednesday, suggesting that the recent boom in artificial intelligence is at the center of the company's strategy. Nvidia shares are up 65% so far in 2023, outpacing the S&P 500 and other semiconductor stocks alike. During Wednesday's fiscal fourth-quarter earnings report, although overall sales declined 21%, investors pushed the stock up about 14% on Thursday, mainly because the company's AI chip business - reported as data centers - rose by 11% to more than $3.6 billion in sales during the quarter, showing continued growth. Nvidia stands to benefit from the AI hype cycle. Now, Stability AI has access to over 5,400 A100 GPUs, according to one estimate from the State of AI report, which charts and tracks which companies and universities have the largest collection of A100 GPUs - although it doesn't include cloud providers, which don't publish their numbers publicly. Brrr." Stability AI is the company that helped develop Stable Diffusion, an image generator that drew attention last fall, and reportedly has a valuation of over $1 billion. "A year ago we had 32 A100s," Stability AI CEO Emad Mostaque wrote on Twitter in January. Some entrepreneurs in the space even see the number of A100s they have access to as a sign of progress. This means that AI companies need access to a lot of A100s. After that, GPUs like the A100 are also needed for "inference," or using the model to generate text, make predictions, or identify objects inside photos. The chips need to be powerful enough to crunch terabytes of data quickly to recognize patterns. Hundreds of GPUs are required to train artificial intelligence models, like large language models. Personal Loans for 670 Credit Score or Lower Personal Loans for 580 Credit Score or Lower Best Debt Consolidation Loans for Bad Credit
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |