Who Is Responsible for AI’s Mistakes?

This post is part of a series that features answers to top AI questions, straight from our data scientists. To receive similar content to your inbox, subscribe to our monthly email digest.

Strategizing, designing, and implementing AI can be a significant investment of resources for organizations, which means AI mistakes can be costly.

To mitigate bias, risk, and other unintended consequences of artificial intelligence, it’s critical to involve humans in every step of the design and development process. 

But when these mistakes do happen, who do we hold accountable? Is the AI at fault, or does the responsibility fall to human teams? 

Our AI Has Failed—Who Do We Blame?

Although your first instinct may be to blame AI, a seemingly foreign and complex blend of algorithms, humans are ultimately responsible for AI’s mistakes. 

The good news, however, is that many AI mistakes are easily preventable. Asking the right questions and having transparent conversations throughout all stages of the AI design process are key to identifying common mistakes before they become problematic.

3 Common Mistakes in AI Design and Development

AI mistakes can take many forms—and some are more common than others. Below are three mistakes organizations often make when diving into AI design and development. 

1. Using Biased Training Data

Bias occurs when results cannot be generalized widely.

“We often think of bias resulting from preferences or exclusions in training data,” said Stanford doctor Dr. Sanjiv M. Narayan in an interview. “But bias can also be introduced by how data is obtained, how algorithms are designed, and how AI outputs are interpreted.”

No matter how the biased data occurs, its inclusion in AI will lead to biased outcomes. Consider these steps to avoid biased data and biased outcomes. 

2. Setting Unclear Objectives

Unclear objectives are the result of an organization that has not defined the problem they are attempting to solve. And when an organization can’t define their problem, it’s unlikely they’ll select the most efficient solution.

AI is built to complete very narrow, structured tasks. It won’t succeed if a clear issue or problem isn’t identified first. Sometimes, once a problem is identified, companies realize that AI isn’t the most feasible solution. They may recognize that they have the tools to solve the problem in their existing tech stack.

3. Establishing Unrealistic Expectations

An AI-powered solution is not magic—and organizations that anticipate such have set unrealistic expectations. The thought of quick, effortless results may entice an organization to rush into an AI solution with little to no consideration of the design and development process.

Instead, consider KPIs that can effectively, and realistically, measure the performance of an artificial intelligence model. Improperly measuring the outcomes of an AI may lead an organization to declare the solution a failure when it’s actually performing well. 

5 Questions to Ask Before Designing AI 

Humans are not perfect, which means artificial intelligence will always be subject to some degree of error. However, the following questions may help identify and prevent a mistake from halting the progress of an AI project. The more complicated the AI, the more critical these questions become. 

1. Who is this project going to affect?
2. What decisions will be made? 
3. What is the potential for disparate impact? 
4. How will you ensure privacy, transparency, and fairness are consistently addressed? 
5. How are you measuring and tracking this information along the way?

Regardless of the size of an organization’s AI project, accepting full responsibility for the mistakes that may occur—and taking the appropriate steps to prevent future mistakes—is critical to avoid unintended consequences of AI.

Find the Answers to Your AI Questions 

Learn more strategies to mitigate AI risk—and discover answers to other readers’ top AI questions—by subscribing to our Voices of Trusted AI monthly digest. It’s a once-per-month email that contains helpful trusted AI resources, reputable information and actionable tips straight from data science experts themselves.

Contributor Nicole Ponstingle McCaffrey is the COO and AI Translator at Pandata.