Deep Learning, from first neuron to transformers.

Deep learning is a subfield of machine learning that uses multi-layer neural networks to learn hierarchical representations directly from data.

The artificial neuron is the basic computational unit: a weighted sum of inputs passed through a non-linearity.

Non-linear activations let networks model complex functions. Without them, stacked affine layers collapse to a single affine map.

A loss function measures how wrong the model is. Training is the process of minimising it.

Backpropagation efficiently computes gradients of the loss with respect to every weight using the chain rule.

Optimizers decide how gradients are turned into weight updates.

Techniques that prevent overfitting so the model performs well on unseen data.

Multi-layer perceptrons are fully connected networks where information flows in one direction from input to output.

CNNs exploit spatial structure using shared filters, making them the backbone of computer vision.

RNNs process sequences by maintaining a hidden state that carries information across time steps.

Gated recurrent units that control what to remember and forget, solving much of the vanishing-gradient problem.

Networks that learn to compress input into a latent code and reconstruct it.

Two networks — a generator and a discriminator — compete, producing highly realistic synthetic data.

Attention-based architectures that process all tokens in parallel; the foundation of modern LLMs and vision models.

GNNs learn on graph-structured data by passing messages between connected nodes.

Reuse knowledge from a model pre-trained on a large dataset to solve a new task with less data.

Generative models that learn to reverse a gradual noising process, powering modern image and video generation.

Combines neural networks with reinforcement learning so agents learn policies from rewards.

A practical checklist for training, debugging and evaluating deep networks.