Learning path / Deep learning
Shape the signal.
Choose what “wrong” means.
Activation functions determine how representations and gradients flow. Objective functions determine what evidence the model tries to explain—and most familiar losses are negative log-likelihoods in disguise. Later lessons build the architectures that stack these pieces at scale.
Small pieces first.
Architectures after.
Each lesson pairs a derivation with an interactive instrument. Planned lessons show where the path grows next; the sequence always reads top to bottom.
How nonlinearity shapes representations, sparsity, saturation, and gradient flow.
Probe the functions →Derive MSE, MAE, and cross-entropy as negative log-likelihoods of observation models.
Follow the derivation →Probe each loss curve to see how its assumptions change penalties and gradients.
Compare the losses →Convolutions, depth, and residual connections as inductive biases.
PlannedGated recurrence and attention as answers to long-range credit assignment.
PlannedStart here
Activation functions
Why stacking linear layers achieves nothing—and how one nonlinear function fixes it. Probe seven activations and their gradients.
Probe the functions