AI from the Ground Up

From a single artificial neuron to the inside of GPT — the real math and the real code, one short lesson at a time. No magic, no hand-waving: every formula explained symbol by symbol, every claim cited, every number computed for real.

Start with Lesson 1 →

Phase 1 — One Neuron

The atom of all modern AI: what it computes and what it can decide.

1. 1The Artificial Neuron
   Weights, bias, sigmoid — what one neuron computes, in math, Swift, and Python.
2. 2The Neuron as Decision-Maker
   One neuron draws a straight line — set its weights by hand to build the AND gate.

Phase 2 — Many Neurons → a Network

Neurons stack into layers, layers chain into networks.

1. 3A Layer of Neurons — Enter the Vector
   Many neurons share the same inputs: the weight matrix and a = σ(W·x + b).
2. 4Stacking Layers — the Forward Pass
   Chain layers and push numbers through; a tiny hand-built network solves XOR.
3. 5Activation Functions — and Why ReLU Took Over
   Why networks need nonlinearity at all; meet sigmoid, tanh, and ReLU.

Phase 3 — How a Network Learns

The heart of the course: loss, gradients, and the loop that trains.

1. 6Loss — Measuring How Wrong We Are
   One number summarizes the network's wrongness — the key that unlocks learning.
2. 7Gradient Descent — Rolling Downhill
   Compute the slope, step against it, watch the loss shrink — by hand and in code.
3. 8Backpropagation — the Error Flows Backwards
   The chain rule as link-multiplication: one full backward pass, every number shown.
4. 9The Training Loop
   Forward, loss, backward, update, repeat — a neuron learns the AND gate itself.

Phase 4 — Build Your Own

The mission cashes out: networks you train yourself.

1. 10Train a Tiny Network from Scratch
   Pure Python, no libraries: a 2-2-1 network learns XOR from data.
2. 11The Same Net in PyTorch
   Rebuild it in ~25 lines in Colab — and map every line to the concept it replaces.
3. 12Real Data — Handwritten Digits
   Train a digit reader on MNIST in Colab. Your own model, on real data.

Phase 5 — Under the Hood of GPT

Text generation from the inside: tokens, attention, the transformer, and how ChatGPT is made.

1. 13Text Becomes Numbers: Tokens
   Build a tokenizer in ten lines; meet BPE and why LLMs never see letters.
2. 14Your First Language Model
   Count, divide, sample — a bigram model invents names, and GPT's job becomes one sentence.
3. 15Embeddings: Words as Vectors
   Meaning becomes geometry: cosine similarity and king − man + woman = queen, by hand.
4. 16Attention: Every Token Looks Back
   Queries, keys, values, causal mask — "cat" absorbs "fluffy" and "blue", every number checked.
5. 17The Transformer
   Assembly day: all the parts snap together into the architecture of GPT. No new math.
6. 18How GPT Is Trained
   Pretraining, RLHF, temperature, hallucination — and training your own GPT in Colab.
7. 19The Practitioner's Toolbox
   Languages, notebooks, GPUs, Hugging Face, Ollama — and what to install for each path.

Appendix — keep these open while you read

Reference pages: printable, themed, always one click away.

1. APython for the Swift Developer
   Every Python construct used in this book, explained through Swift. Read it before Lesson 1's Python tab.
2. BFormulas Cheat Sheet
   All the course math on one printable page, with verified sanity-check numbers.