Episode 11 · Planned
Model internals: weights to optimization
Trace weights, attention, gradients and optimizer state with a small inspectable model.
Use a small inspectable model to trace stored weights through embeddings, attention, the forward pass and logits. Then trace the backward pass through gradients, optimizer state and parameter updates. Connect these mechanisms to memory and compute costs without presenting the small-model exercise as a large-model benchmark.
Evidence: reproducible tensor-shape and parameter-count walkthroughs, attention inspection, gradient checks and optimizer-state accounting.