The Cost of AI

It's Not Free Magic

Using ChatGPT or Gemini feels like magic, but behind the scenes, massive industrial machinery is turning.
Every time you ask a question, a supercomputer somewhere heats up.

The Currency: Tokens

AI doesn't read words; it reads Tokens.

• A token is roughly 3/4 of a word.
• "Hello" = 1 token.
• "Hamburger" = 3 tokens (Ham-bur-ger).

AI companies charge by the token. It's like paying for a telegram by the letter.

• Input Cost: What you type.
• Output Cost: What the AI writes (usually more expensive).

The Engine: GPUs

Traditional software runs on CPUs (Central Processing Units).
AI runs on GPUs (Graphics Processing Units).
These are the same chips used for video games, but much more powerful. A single high-end AI chip (like an NVIDIA H100) can cost as much as a luxury car ($30,000+).
To train a model like GPT-4, companies use thousands of these chips for months.

The Energy Bill

All those chips use electricity. Training a large AI model can use as much energy as a small town uses in a year.
This is why "Green AI" (making AI more efficient) is a huge topic of research.

Conclusion

AI is a resource-intensive technology. It requires expensive hardware, massive amounts of electricity, and a new way of thinking about cost (per-token).