Foundations / What is Deep Learning?
What is Deep Learning?
BeginnerDeep learning is a subfield of machine learning that uses multi-layer neural networks to learn hierarchical representations directly from data.
AI, ML and DL
Artificial Intelligence is the broad goal of building systems that act intelligently. Machine Learning is a subset where systems learn patterns from data instead of following hand-written rules. Deep Learning is a subset of ML that uses neural networks with many layers.
| Aspect | Classical ML | Deep Learning |
|---|---|---|
| Features | Hand-engineered | Learned automatically |
| Data needs | Small to medium | Large datasets |
| Compute | CPU is often enough | GPUs / TPUs |
| Interpretability | Higher | Lower (black box) |
Why deep learning works now
- Big data: internet-scale images, text and audio.
- Hardware: GPUs make massive parallel matrix math cheap.
- Algorithms: ReLU, better initialisation, normalisation, residual connections, Adam.
- Software: frameworks such as PyTorch, TensorFlow and JAX.
Learning paradigms
- Supervised learning — learn from labelled input/output pairs (classification, regression).
- Unsupervised learning — find structure in unlabelled data (clustering, autoencoders).
- Self-supervised learning — create labels from the data itself (next-token prediction, masked modelling).
- Reinforcement learning — learn by interacting with an environment and receiving rewards.
Interview questions & answers
Q1. How does deep learning differ from classical machine learning?
Answer: Deep learning uses multi-layer neural networks to learn representations from data, reducing the need for hand-engineered features. It often benefits from large datasets and substantial compute, while classical methods can be more effective for small datasets or structured tabular data.
Q2. What is the difference between supervised and self-supervised learning?
Answer: Supervised learning uses externally supplied target labels. Self-supervised learning creates training targets from the data itself, such as predicting masked words or the next token. Both optimize a prediction objective, but the source of supervision differs.
Q3. Why are deep networks useful for images and language?
Answer: Multiple layers can learn increasingly abstract representations: edges and shapes in images, or contextual relationships in text. Appropriate architectures build useful assumptions into the model, such as spatial locality in CNNs and token interactions in transformers.