But the same qualities that make those graphics processor chips, or GPUs, so effective at creating powerful AI systems from scratch make them less efficient at putting AI products to work.
That’s opened up the AI chip industry to rivals who think they can compete with Nvidia in selling so-called AI inference chips that are more attuned to the day-to-day running of AI tools and designed to reduce some of the huge computing costs of generative AI.
“These companies are seeing opportunity for that kind of specialized hardware,” said Jacob Feldgoise, an analyst at Georgetown University’s Center for Security and Emerging Technology. “The broader the adoption of these models, the more compute will be needed for inference and the more demand there will be for inference chips.”
WHAT IS AI INFERENCE?
It takes a lot of computing power to make an AI chatbot. It starts with a process called training or pretraining — the “P” in ChatGPT — that involves AI systems “learning” from the patterns of huge troves of data. GPUs are good at doing that work because they can run many calculations at a time on a network of devices in communication with each other.
However, once trained, a generative AI tool still needs chips to do the work — such as when you ask a chatbot to compose a document or generate an image. That’s where inferencing comes in. A trained AI model must take in new information and make inferences from what it already knows to produce a response.
GPUs can do that work, too. But it can be a bit like taking a sledgehammer to crack a nut.
“With training, you’re doing a lot heavier, a lot more work. With inferencing, that’s a lighter weight,” said Forrester analyst Alvin Nguyen.
この記事は Techlife News の November 23, 2024 版に掲載されています。
7 日間の Magzter GOLD 無料トライアルを開始して、何千もの厳選されたプレミアム ストーリー、9,000 以上の雑誌や新聞にアクセスしてください。
すでに購読者です ? サインイン
この記事は Techlife News の November 23, 2024 版に掲載されています。
7 日間の Magzter GOLD 無料トライアルを開始して、何千もの厳選されたプレミアム ストーリー、9,000 以上の雑誌や新聞にアクセスしてください。
すでに購読者です? サインイン
AUSTRALIA PLANS TO TAX DIGITAL PLATFORMS THAT DON'T PAY FOR NEWS
The Australian government said it will tax large digital platforms and search engines unless they agree to share revenue with Australian news media organizations.
JAPAN'S NISSAN RESHUFFLES MANAGEMENT TO FIX ITS MONEY-LOSING BUSINESS
Embattled Japanese automaker Nissan has tapped Jeremie Papin, who was overseeing its U.S. operations, as its chief financial officer in a major management reshuffle billed as key to a turnaround.
EPA AWARDS $135 MILLION TO CALIFORNIA TO PHASE OUT BIG DIESEL TRUCKS
The Environmental Protection Agency is awarding $135 million in grants to fund 13 projects in California to help the state wean off fossil fuels and phase out big rigs that run on diesel.
NEARLY HALF OF US TEENS ARE ONLINE 'CONSTANTLY,' PEW REPORT FINDS
Nearly half of American teenagers say they are online “constantly” despite concerns about the effects of social media and smartphones on their mental health, according to a new report published by the Pew Research Center.
OPENAI'S LEGAL BATTLE WITH ELON MUSK REVEALS INTERNAL TURMOIL OVER AVOIDING AI 'DICTATORSHIP'
A 7-year-old rivalry between tech leaders Elon Musk and Sam Altman over who should run OpenAI and prevent an artificial intelligence “dictatorship” is now heading to a federal judge as Musk seeks to halt the ChatGPT maker’s ongoing shift into a for-profit company.
TECH TIP: HOW TO PROTECT YOUR COMMUNICATIONS THROUGH ENCRYPTION
After a sprawling hacking campaign exposed the communications of an unknown number of Americans, U.S. cybersecurity officials are advising people to use encryption in their communications.
TRUMP HOSTS APPLE CEO AT MAR-A-LAGO AS BIG TECH LEADERS CONTINUE OUTREACH TO PRESIDENT-ELECT
Donald Trump hosted Apple CEO Tim Cook for a Friday evening dinner at the president-elect's Mar-a-Lago resort, according to a person familiar with the matter who was not authorized to comment publicly.
MUSK SAYS US IS DEMANDING HE PAY PENALTY OVER DISCLOSURES OF HIS TWITTER STOCK PURCHASES
Elon Musk says the Securities and Exchange Commission wants him to pay a penalty or face charges involving what he disclosed or failed to disclose - about his purchases of Twitter stock before he bought the social media platform in 2022.
ELON MUSK WANTS TO TURN SPACEX'S STARBASE SITE INTO A TEXAS CITY
SpaceX is launching a new mission: making its Starbase site a new Texas city.
OPENAI RELEASES AI VIDEO GENERATOR SORA BUT LIMITS HOW IT DEPICTS PEOPLE
OpenAl has publicly released its new artificial intelligence video generator Sora but the company won't let most users depict people as it monitors for patterns of misuse.