Deep Learning
Machine learning with many-layered neural networks — the technique behind modern AI.
Machine learning using neural networks with many layers. Each layer learns progressively more abstract features, which is why a deep network can go from raw pixels to "that is a cat" without anyone specifying edges or whiskers.
The "deep" is about depth of composition, not depth of thought. What changed in the 2010s was scale: bigger networks, more data, and GPUs to train them.
Depth is what buys abstraction without hand-designed features. In older vision systems somebody wrote an edge detector; in a deep network the early layers effectively discover one because edges are useful for the layers above. That is the whole argument for stacking layers, and it is why "more data" became a strategy.
Why nobody writes feature extractors any more
Predicting churn from raw support tickets, years before LLMs.
The old pipeline had a hand-built feature stage — counts, sentiment words, escalation flags. Fine-tuning a model on the raw text beats it, because the model finds features nobody thought to enumerate. It also means you cannot read the feature list to explain a decision.
- guideLLMs: what's a large language model? ↗
Encoder, decoder and decoder-only designs, self-attention, parameters and token-by-token generation, explained from the ground up.
- paperDistilling the knowledge in a neural network ↗
The original distillation idea: train a small model to match a larger one’s soft outputs. Foundational for the concept, distinct from later LLM-specific student/teacher recipes.
Neural Network
A stack of simple units whose weighted connections are learned from data.
Machine Learning · ML
Programs that improve from examples instead of explicit instructions.
Transformer
The 2017 network architecture that all modern language models are built on.
Training
The expensive one-off process that sets the model’s parameters.