Deep Learning 2026
Laboratory material for the Deep Learning 2026 course, organised into four units and numbered by topic in teaching order. Each topic is a Jupyter notebook that opens in Colab; selected topics also have an interactive companion page that runs in the browser.
Where PyTorch came from and how it is put together, then tensors worked through from first principles. Ships with a completed answer notebook.
torch history · tensors · dtypes · indexing · broadcasting
A single neuron always draws a straight line, so it cannot solve XOR — the argument for hidden layers, followed by shallow networks on 1D inputs.
XOR · decision boundary · hidden units · 1D inputs
One tiny 1→3→1 network with a different activation plugged in each time, so every change in the output is attributable to the activation alone.
ReLU · sigmoid · tanh · forward prop · loss
The forward pass and its loss built by hand on the Bank Customer Churn dataset. Student notebook with blanks, plus the answer key.
manual forward pass · weights · bias · BCE loss
Three losses implemented from their definitions and checked against PyTorch, with the rule for choosing between them stated and applied.
MSE · MAE · BCE · cross-entropy · reduction
Why hand-differentiation does not scale, then the computational graph that replaces it: recorded operations walked backward by the chain rule.
requires_grad · computational graph · .backward() · chain rule
A binary classifier assembled from PyTorch's core building blocks, then the same loop rewritten around nn.Module.
training loop · nn.Module · optimiser · zero_grad · evaluation
The same network trained three times, changing only batch_size, with the loss curve of each regime recorded and read against the theory.
batch GD · stochastic GD · mini-batch · loss curves
Six optimizers run on one ill-conditioned ravine, each answering the failure of the one before it, with every trajectory and loss curve read off the same surface.
SGD · momentum · NAG · AdaGrad · RMSProp · Adam
A custom Dataset written from its three methods and served by a DataLoader, on ten samples small enough to check every batch by eye.
__len__ · __getitem__ · batching · shuffling · collate
A two-block CNN trained on CIFAR-10, then rebuilt with BatchNorm and Dropout, with train and test accuracy compared to show the overfitting gap closing.
Conv2d · MaxPool2d · output shapes · BatchNorm2d · Dropout · CIFAR-10
A baseline CNN, AlexNet, a VGG-style network and ResNet-18 trained with one shared function — parameters move from the FC head into the conv layers as test accuracy climbs from 72% to 83%.
AlexNet · VGG · ResNet · skip connections · global average pooling
Batch, stochastic and mini-batch gradient descent compared in PyTorch on a purchase-prediction dataset. The update rule is developed first, then traced through a complete training loop, with the loss curve of each regime recorded and read against the theory.
batch GD · stochastic GD · mini-batch · loss curves · optimisation path · vectorisation vs. memory
Why a full dataset cannot be passed to the model in one step, and how the two abstractions divide the work. A Dataset is built from its three required methods, then a DataLoader is configured around it.
Dataset · DataLoader · transforms · samplers · collate_fn · num_workers