Training cost: what $100, $10,000 and $100 million buy in AI

What training cost is

Training cost is the bill for teaching an AI model from scratch. The standard way to estimate it counts one thing: how long you rent powerful computer chips, called GPUs, and at what price. It sounds like a trade secret, but four public inputs are enough to work it out.

What the video shows

The video gives three reference points:

• About $10,000 rents enough computing power to train a model with about 5.5 billion parameters (the adjustable numbers inside a model that training tunes) in around two weeks.
• Past roughly $100,000, the limit stops being money and becomes access to enough chips.
• A model that matches GPT-2, OpenAI's 2019 language model, now costs under $100 to train.

An everyday example

Imagine pricing a big print job at a shop that rents out machines by the hour. You need the number of pages, how many pages a machine really prints per hour (fewer than the box promises, once you count paper jams), and the hourly price. None is secret, and if the price rises, you know which number to change. A small job needs one machine. A job that needs ten thousand machines at once is a different problem: no shop has that many to rent.

In AI training, the pages are the calculations and the machines are chips.

How the estimate works

The course builds its estimate from four inputs, each with a source:

| Input | Value | What it means | |---|---|---| | Chip speed | 989.5 trillion calculations per second | An NVIDIA H100's real peak. The advertised figure is double, because it assumes a shortcut (sparsity) that training does not use. | | Real use | 40% | The share of that peak a real run actually achieves. Meta reported 38-43% for Llama 3. | | Price | $3.99 per GPU-hour | What it costs to rent one chip for one hour, at a public on-demand price checked in September 2026. | | Data | 20 tokens per parameter | The Chinchilla ratio, a common rule of thumb: about 20 chunks of text (tokens) for every parameter. |

Multiply them out and you get a ladder:

• $100: about 546 million parameters, roughly 3 hours on one rented machine with 8 GPUs.
• $10,000: about 5.5 billion parameters, about 13 days on that same machine.
• $100,000: about 17 billion parameters: 131 days on one machine, or 12 GPUs for three months.
• $100 million: about 546 billion parameters, which needs around 11,600 H100s wired together for three months.

That explains the video's second point. Nobody rents nearly twelve thousand connected chips with a credit card. Beyond about $100,000, the hard part is getting a cluster, meaning a large group of linked chips, not paying for it.

Does the napkin math hold up?

Checked against runs with published numbers, it is off by about 15% or less. One public project trained a 1.6-billion-parameter copy of GPT-2 on one 8-GPU machine in 24 hours for $672. The formula says $672 buys about 1.4 billion parameters; the real run trained 1.56 billion. For Meta's Llama 3 405B, it predicts 26.67 million GPU-hours against the 30.84 million Meta published.

A common misconception

You may have read that GPT-2 cost about $43,000 to train in 2019 and costs tens of dollars today. The modern half holds up: Karpathy's nanochat project reports GPT-2-level results for about $48, assuming $3 per GPU-hour, or nearer $64 at the $3.99 list price. The 2019 half does not. OpenAI never published a training time or a cost. The $43,000 is an hourly hardware price multiplied by an assumed week of training that nobody ever sourced.

Why it matters

Most training costs in the news are estimates, not receipts. The GPT-3 paper gives no dollar figure, and neither does the Llama 3 paper. Every number you read was built from a count of calculations, a hardware assumption and a price assumption. So the useful question is always: whose assumptions, and are they still current?

Learn it step by step in Chapter 10 of our free course AI From Scratch: Pretraining an LLM: Data, Compute, Scaling Laws and Cost.