Neural Networks and Deep Learning in Plain Words

A neural network borrows a loose idea from the brain: lots of simple units, each passing a signal to the next, with the strength of each connection adjustable. The resemblance to a real brain is thin. A real neuron is a living cell with chemistry nobody fully understands. An artificial neuron is a line of multiplication. But stack enough of them and train them well, and they recognize faces, transcribe speech and write essays.

One neuron, by hand

An artificial neuron works in steps:

1. Multiplies each input by its weight (how much that input matters).
2. Adds those up, plus one extra number called the bias (a baseline nudge).
3. Passes the total through a squashing function, so the output lands in a useful range.

Notice the negative weight on "in your contacts." Nobody had to type a rule saying "trust contacts." If training found that emails from contacts were rarely spam, it would push that weight negative on its own.

Layers: neurons feeding neurons

One neuron can only draw one straight dividing line through the data. The trick is to wire many together in layers. The first layer takes the raw inputs. Each later layer takes the outputs of the layer before as its inputs. The last layer gives the answer.

Count the parameters in the picture. Three inputs to four neurons: 12 weights plus 4 biases. Four to four: 16 weights plus 4 biases. Four to one: 4 weights plus 1 bias. Total 41 numbers. The 2012 AlexNet image model had 60 million. OpenAI's GPT-3 had 175 billion. Meta's open Llama 4 Maverick, released in April 2025, has 400 billion in total, of which 17 billion are used for any one word (a design called mixture of experts, where only part of the network wakes up each time).

Why depth helps

Researchers who peer inside image networks find a rough division of labor. Early layers respond to edges and color patches. Middle layers combine edges into textures and shapes: an eye, a wheel, fur. Late layers combine those into whole things: a face, a car, a cat. Nobody assigned those jobs. They emerge from training because they help lower the loss. That stacking of simple patterns into complex ones is why many layers (deep learning) beat a few.

How it learns: gradient descent

Training a network is the same loop as lesson 3: guess, measure the loss, nudge the weights, repeat. The question is which way to nudge each of billions of weights.

Picture yourself on a foggy hillside at night, trying to reach the lowest point in the valley. You can't see the valley. You can feel the slope under your boots. So you take a small step downhill, feel again, step again. That is gradient descent. The height is the loss. Your position is the current set of weights. The slope under your feet is the gradient: for every weight, which way is downhill and how steep.

The step size matters. Set it to 0.5 instead of 0.05 and the first step would jump the slope from 15 to 48, far past the bottom, with a much bigger loss than where it started. Too small, like 0.0001, and you'd need tens of thousands of steps. Engineers spend real time tuning this.

Backpropagation

With one adjustable number, finding the gradient is easy. With billions spread across dozens of layers, you need bookkeeping. Backpropagation, popularized by the 1986 Rumelhart, Hinton and Williams paper, works backward from the error. The output was too high by this much; this last-layer weight contributed this share; the neuron feeding it got this share of blame; and so on back to the first layer. Each weight gets its own nudge in one backward sweep. Modern chips called GPUs are good at the huge piles of multiplication this needs, which is why the 2012 result used graphics cards and why AI companies now buy them by the hundred thousand.