The Working Limitations of Large Language Models
MIT Sloan Management Review|Winter 2024
Overestimating the capabilities of Al models like ChatGPT can lead to unreliable applications.
Mikhail Burtsev, Martin Reeves, and Adam Job
The Working Limitations of Large Language Models

Large language models (LLMs) seem set to transform businesses. Their ability to generate detailed, creative responses to queries in plain language and code has sparked a wave of excitement that led ChatGPT to reach 100 million users faster than any other technology after it first launched. Subsequently, investors poured over $40 billion into artificial intelligence startups in the first half of 2023 — more than 20% of all global venture capital investments — and companies from seedstage startups to tech giants are developing new applications of the technology.

But while LLMs are incredibly powerful, their ability to generate humanlike text can invite us to falsely credit them with other human capabilities, leading to misapplications of the technology. With a deeper understanding of how LLMs work and their fundamental limitations, managers can make more informed decisions about how LLMs are used in their organizations, addressing their shortcomings with a mix of complementary technologies and human governance.

The Mechanics of LLMs 

An LLM is fundamentally a machine learning model designed to predict the next element in a sequence of words. Earlier, more rudimentary language models operated sequentially, drawing from a probability distribution of words within their training data to predict the next word in a sequence. (Think of your smartphone keyboard suggesting the next word in a text message.) However, these models lack the ability to consider the larger context in which a word appears and its multiple meanings and associations.

This story is from the {{IssueName}} edition of {{MagazineName}}.

Start your 7-day Magzter GOLD free trial to access thousands of curated premium stories, and 9,000+ magazines and newspapers.

This story is from the {{IssueName}} edition of {{MagazineName}}.

Start your 7-day Magzter GOLD free trial to access thousands of curated premium stories, and 9,000+ magazines and newspapers.

MORE STORIES FROM MIT SLOAN MANAGEMENT REVIEWView all
Avoiding Harm in Technology Innovation
MIT Sloan Management Review

Avoiding Harm in Technology Innovation

To capitalize on emerging technologies while mitigating unanticipated consequences, innovation managers need to establish a systematic review process.

time-read
10+ mins  |
Fall 2024
Make a Stronger Business Case for Sustainability
MIT Sloan Management Review

Make a Stronger Business Case for Sustainability

When greener products and processes add costs, managers can shift other levers to maintain profitability.

time-read
9 mins  |
Fall 2024
How to Turn Professional Services Into Products
MIT Sloan Management Review

How to Turn Professional Services Into Products

Product-based business models can help services firms achieve greater scale and profitability. But the transformation can be challenging.

time-read
10 mins  |
Fall 2024
Do You Really Need a Chief AI Officer?
MIT Sloan Management Review

Do You Really Need a Chief AI Officer?

The right answer depends on the strategic importance and maturity of AI in your company.

time-read
10+ mins  |
Fall 2024
Where To Next? Opportunity on the Edge
MIT Sloan Management Review

Where To Next? Opportunity on the Edge

Doing business in regions considered less stable or developed can pay off for companies. But they must invest in working with local communities.

time-read
10 mins  |
Fall 2024
Make Smarter Investments in Resilient Supply Chains
MIT Sloan Management Review

Make Smarter Investments in Resilient Supply Chains

Many companies invest in resilience only after a disruption. Applying the concept of real options can help decision makers fortify supply chain capabilities no matter the crisis.

time-read
10+ mins  |
Fall 2024
The Three Traps That Stymie Reinvention
MIT Sloan Management Review

The Three Traps That Stymie Reinvention

Organizational identity, architecture, and collaboration can be either assets or liabilities to pursuing growth in new sectors.

time-read
10+ mins  |
Fall 2024
What Makes Companies Do the Right Thing?
MIT Sloan Management Review

What Makes Companies Do the Right Thing?

Vaccine makers varied widely in their engagement with global public health efforts to broaden access to COVID-19 immunizations. Ethically motivated leadership was a dominant factor.

time-read
10+ mins  |
Fall 2024
Build the Right C-Suite Team for Your Strategy
MIT Sloan Management Review

Build the Right C-Suite Team for Your Strategy

CEOs can foster a more effective leadership team by understanding when to tap senior executives' competitive instincts and when to encourage collaboration.

time-read
10+ mins  |
Fall 2024
A Better Way to Unlock Innovation and Drive Change
MIT Sloan Management Review

A Better Way to Unlock Innovation and Drive Change

A strengths-based approach to building teams can win employee commitment to change and foster an inclusive, agile culture.

time-read
10+ mins  |
Fall 2024