Pandata Blog

Approachable, Ethical, Human-Centered AI

AI compute

What Is Compute in AI? 3 Key Challenges

In its simplest form, compute describes the manipulation of information; it’s used to organize and process data. Machines that use compute to perform tasks have three primary components: logic, memory, and interconnect.

In the logic component, operations are conducted using information stored in the memory component; the interconnect component serves as an input or holding point for information that either was processed or will be processed. 

Compute, and the conversation surrounding its uses in AI, is growing in popularity. But with its increasing applications come more challenges—and data scientists have yet to provide definite solutions to many of them.

How Compute Is Used in AI

AI in the modern age is accelerated by three factors: innovation in algorithms, data, and compute. However, compute is often considered the most impactful—and the easiest to measure. 

Compute is used in AI for training and inference purposes. In the training stage, compute is used to determine the significance of weights and biases in a network. Once an AI is fully trained, the compute that was used to power its final training “run,” or processing of data, is often considered representative of the AI’s overall capabilities. 

A trained network can then use compute to produce an output via the inference process. To produce an output via inference, an AI only needs one input. Consider, for example, how you would ask Siri, a trained artificial intelligence, for directions. The input would be your request, and Siri would produce the output: directions to your destination. 

3 Key Challenges of Compute in AI

While research around compute offers exciting opportunities for AI development, it also poses a number of unique challenges. 

1. Biased Outputs

Training an AI is a repetitive process that 1) requires the use of compute and 2) relies on accurately labeled data. Otherwise, users confront the ever-present risk of biased outputs. But because compute exists on one axis (think: we can only ever use more or less compute), there’s no concrete way its use can reduce bias.

2. Model Theft 

Working to train a model from scratch requires significantly more compute than using a  foundational model to produce an output. The availability of this shortcut often leads to model-theft, where individuals can duplicate a target model’s functionality by using predicted outputs of different inputs.

3. Increasing Costs

The use of compute to scale AI has outpaced human cognition. Put simply, compute has allowed AI to learn on its own faster than humans can teach an AI new knowledge. Since 2012, the amount of compute used in AI has doubled every 6.2 months. This level of growth, and associated monetary costs, are not sustainable.

Exploring Solutions to Compute Challenges

Data scientists have not yet created concrete solutions to these compute challenges, but many opportunities are open for exploration. Here are just a few:

  • Increase the rate of algorithm innovation. 
  • Investigate how better measurements of efficiency can lead to better cost predictability. 
  • Focus less on compute spending ability and more on diverse, interdisciplinary research teams.

The future of compute in AI is sure to bring exciting new developments across AI-adopting industries—ones that will certainly further our mission to develop human-centered, trustworthy AI.

Get Actionable Insights on Current AI Topics

Compute in AI is just one of many evolving conversations around trustworthy AI. Stay up-to-date on the latest in trusted AI and data science by subscribing to our Voices of Trusted AI monthly digest. It’s a once-per-month email that contains helpful trusted AI resources, reputable information, and actionable tips.

Trusted AI

Contributor: Nicole McCaffrey is the COO and AI Translator at Pandata.