Auditing Algorithmic Risk
MIT Sloan Management Review|Summer 2024
How do we know whether algorithmic systems are working as intended? A set of simple frameworks can help even nontechnical organizations check the functioning of their AI tools.
Cathy O’Neil, Jake Appel, and Sam Tyner-Monroe
Auditing Algorithmic Risk

ARTIFICIAL INTELLIGENCE, LARGE LANGUAGE MODELS (LLMs), and other algorithms are increasingly taking over bureaucratic processes traditionally performed by humans, whether it’s deciding who is worthy of credit, a job, or admission to college, or compiling a year-end review or hospital admission notes.

But how do we know that these systems are working as intended? And who might they be unintentionally harming? Given the highly sophisticated and stochastic nature of these new technologies, we might throw up our hands at such questions. After all, not even the engineers who build these systems claim to understand them entirely or to know how to predict or control them. But given their ubiquity and the high stakes in many use cases, it is important that we find ways to answer questions about the unintended harms they may cause. In this article, we offer a set of tools for auditing and improving the safety of any algorithm or AI tool, regardless of whether those deploying it understand its inner workings.

Algorithmic auditing is based on a simple idea: Identify failure scenarios for people who might get hurt by an algorithmic system, and figure out how to monitor for them. This approach relies on knowing the complete use case: how the technology is being used, by and for whom, and for what purpose. In other words, each algorithm in each use case requires separate consideration of the ways it can be used for — or against — someone in that scenario.

This applies to LLMs as well, which require an application-specific approach to harm measurement and mitigation. LLMs are complex, but it’s not their technical complexity that makes auditing them a challenge; rather, it’s the myriad use cases to which they are applied. The way forward is to audit how they are applied, one use case at a time, starting with those in which the stakes are highest.

هذه القصة مأخوذة من طبعة Summer 2024 من MIT Sloan Management Review.

ابدأ النسخة التجريبية المجانية من Magzter GOLD لمدة 7 أيام للوصول إلى آلاف القصص المتميزة المنسقة وأكثر من 9,000 مجلة وصحيفة.

هذه القصة مأخوذة من طبعة Summer 2024 من MIT Sloan Management Review.

ابدأ النسخة التجريبية المجانية من Magzter GOLD لمدة 7 أيام للوصول إلى آلاف القصص المتميزة المنسقة وأكثر من 9,000 مجلة وصحيفة.

المزيد من القصص من MIT SLOAN MANAGEMENT REVIEW مشاهدة الكل
Avoiding Harm in Technology Innovation
MIT Sloan Management Review

Avoiding Harm in Technology Innovation

To capitalize on emerging technologies while mitigating unanticipated consequences, innovation managers need to establish a systematic review process.

time-read
10+ mins  |
Fall 2024
Make a Stronger Business Case for Sustainability
MIT Sloan Management Review

Make a Stronger Business Case for Sustainability

When greener products and processes add costs, managers can shift other levers to maintain profitability.

time-read
9 mins  |
Fall 2024
How to Turn Professional Services Into Products
MIT Sloan Management Review

How to Turn Professional Services Into Products

Product-based business models can help services firms achieve greater scale and profitability. But the transformation can be challenging.

time-read
10 mins  |
Fall 2024
Do You Really Need a Chief AI Officer?
MIT Sloan Management Review

Do You Really Need a Chief AI Officer?

The right answer depends on the strategic importance and maturity of AI in your company.

time-read
10+ mins  |
Fall 2024
Where To Next? Opportunity on the Edge
MIT Sloan Management Review

Where To Next? Opportunity on the Edge

Doing business in regions considered less stable or developed can pay off for companies. But they must invest in working with local communities.

time-read
10 mins  |
Fall 2024
Make Smarter Investments in Resilient Supply Chains
MIT Sloan Management Review

Make Smarter Investments in Resilient Supply Chains

Many companies invest in resilience only after a disruption. Applying the concept of real options can help decision makers fortify supply chain capabilities no matter the crisis.

time-read
10+ mins  |
Fall 2024
The Three Traps That Stymie Reinvention
MIT Sloan Management Review

The Three Traps That Stymie Reinvention

Organizational identity, architecture, and collaboration can be either assets or liabilities to pursuing growth in new sectors.

time-read
10+ mins  |
Fall 2024
What Makes Companies Do the Right Thing?
MIT Sloan Management Review

What Makes Companies Do the Right Thing?

Vaccine makers varied widely in their engagement with global public health efforts to broaden access to COVID-19 immunizations. Ethically motivated leadership was a dominant factor.

time-read
10+ mins  |
Fall 2024
Build the Right C-Suite Team for Your Strategy
MIT Sloan Management Review

Build the Right C-Suite Team for Your Strategy

CEOs can foster a more effective leadership team by understanding when to tap senior executives' competitive instincts and when to encourage collaboration.

time-read
10+ mins  |
Fall 2024
A Better Way to Unlock Innovation and Drive Change
MIT Sloan Management Review

A Better Way to Unlock Innovation and Drive Change

A strengths-based approach to building teams can win employee commitment to change and foster an inclusive, agile culture.

time-read
10+ mins  |
Fall 2024