The nine parts of the AI pipeline

Area 1 Energy

AI runs in data centers, buildings full of computers that need electricity to run and more electricity to stay cool. The International Energy Agency (IEA) estimated that data centers, which run AI alongside everything else online, used about 1.5% of the world’s electricity in 2024. It projected that their use would roughly double by 2030.

Area 2 Chips

Most AI work runs on graphics processing units (GPUs), chips first built for computer graphics, and on newer chips designed for AI. Their strength is doing an enormous number of simple multiplications at the same time, which is most of what a model does.

A fast chip isn’t enough on its own. It needs fast memory beside it to hold the numbers it is working on. It also needs fast links to other chips, because a large model is spread across many of them. Around all that sit the servers and the cooling that keep it running.

Area 3 Connecting the model to your work

A model on its own takes text in and gives text out. Most useful AI products wrap other software around it. That software can let the model search your own documents before it answers, or use tools such as a database or a web browser. When the software lets the model work through a task in many steps on its own, people call it an agent.

Inside a company, a gateway can sit between the apps and the model providers. It controls who may call which model and keeps a record of every call.

Area 4 Verifying the output

Someone has to decide whether the output is good enough. Labs test models on standard sets of questions, called benchmarks, before release. Companies then test them on their own tasks, and for many uses a person still reads the output before anyone acts on it. In regulated work, such as making medicines, that check is a formal, written step with rules of its own. I wrote about how much of it an AI system needs.

Area 5 Putting the model to work

Using a trained model is called inference. When you ask a question, the model runs once for each token it writes, so a long answer means many runs in a row. Training, area 8, is a one-time cost for each model. Inference is paid every time anyone uses it, for as long as the model stays in service.

Big models run on servers in data centers, usually reached over the internet. Smaller models can also run on a laptop or a phone, which keeps the data on the device.

Area 6 How people actually use it

This area is people putting AI to work. It covers which tasks they hand over and how the work around those tasks changes. In my view, this is where a capable model most often ends up producing nothing useful, because the work around it never changed.

Area 7 Where the data comes from

A model learns from examples. For a large language model (LLM), the kind behind chat assistants, that mostly means text: web pages, books, code and forum posts. Some of the text now comes from other models. People rating the model’s answers add more later, during tuning.

Public text is not unlimited. Epoch AI, a research group that tracks AI trends, estimated in 2024 that the usable stock of public human-written text was about 300 trillion tokens (a token is a word or part of a word). If trends held, models would use all of it at some point between 2026 and 2032.

Area 8 Who’s training models for pharma

Training is where data and chips meet. In the first stage, called pretraining, the model reads a vast amount of text and practices one task over and over: guessing the next token. After each batch of guesses, its internal numbers are nudged a little toward better guesses. Repeat that enough times and the model has absorbed a great deal about language and the world.

The second stage tunes the model to be useful, so it follows instructions and declines some requests. Most of this comes from two sources: example answers written by people, and feedback on the model’s own answers.

All of it takes a lot of computing, and the amount has climbed fast. Epoch AI found that the computing used to train leading models grew four to five times a year from 2010 to May 2024.

Area 9 What the model actually is

What comes out of training is a file. It is a very large one, holding billions of numbers called weights, which set how the model turns what you type into what it writes back. Most of today’s language models use a design called the transformer, first described in a 2017 research paper.

Some companies keep their weights private and let you use the model only through their own service. Others publish the weights so anyone can download and run the model; these are called open-weight models. Models also come in sizes. A smaller one costs less to run but is usually less capable.