This post is part of my ongoing series on the history and evolution of AI. I'll refer back to it whenever neural networks come up in future posts - think of it as a shared reference point for the concepts I keep building on. If you already know how these things work, feel free to skip it. But if you've ever wondered what's actually happening inside an AI model, read on.
The Architecture: Layers and Neurons
A neural network is built from neurons, small computational units connected to other neurons. Each neuron receives inputs, processes them, and passes the result forward to the next set of neurons. Stack enough layers of neurons on top of each other, and you have what's known as a deep neural network. The "deep" in deep learning simply refers to this depth of layers, not to any particular sophistication or mystery beyond that.
Neurons are organised into layers. The input layer is where data enters the network, this could be a piece of text, an image, a sound recording, or a video. The raw data gets converted into a series of numbers before entering the network (I'll skip the technical details of how that conversion works, but numbers are what land in the input layer).
At the other end sits the output layer: the final result. Depending on the network's design, this might be a single neuron (is this a picture of a cat? Yes or no, 1 or 0), or a whole group of neurons producing, say, a paragraph of text or a generated image. Between input and output lie the hidden layers, sometimes just a few, sometimes hundreds or thousands of them. Neurons within a layer don't connect to each other; they only connect forward to the next layer and backward to the previous one. Information flows in one direction: from input, through each hidden layer in turn, and out through the output layer.
What a Neuron Actually Does
Each neuron receives a set of numbers from the neurons connected to it - typically values between 0 and 1. If a neuron has ten connections, it receives ten numbers.
Here's what it does with them: it multiplies each incoming number by a corresponding weight. The weight represents the strength of that connection. A weight of 0 means no connection at all; higher values mean stronger connections. After multiplying each input by its weight, the neuron sums all the results together, then adds one more constant value called the bias. That's it, a handful of multiplications and additions.
Before passing the result forward, the neuron runs it through one more operation: an activation function. This introduces a small mathematical transformation that makes the network capable of learning complex, non-linear patterns. There are many types of activation functions, and choosing between them is one of the many design decisions in building a neural network - I'll leave the details here for now.
The final value is the neuron's output, passed forward to the neurons in the next layer, where the whole process repeats.
The Forward Pass: From Input to Output
Step back and look at the full picture. Data enters as numbers. At each neuron, those numbers are multiplied by weights, a bias is added, and the result passes through an activation function. The transformed values then travel to the next layer, where the same process happens again - and again, and again, layer by layer, until the final output emerges at the other end.
This journey from input to output is called the forward pass.
A useful way to think about it: imagine the weights and biases as dials you can turn. Change a dial, and the output changes. The entire network, however large, is a complex function that takes numbers in and produces numbers out, shaped entirely by the current settings of those dials. And by now you're probably wondering: how do you find the right settings?
The Loss Function: Measuring How Wrong We Are
If we already know what the correct output should be for a given input, we can measure how wrong the network is. That measure is called the loss.
Here's a concrete example. Imagine we feed the network 1,000 images, of which 500 show cats and 500 don't - and we've labelled each one accordingly. We run all 1,000 through the network, and for each image it produces a probability: say, "5% chance this is a cat." If the label says it *is* a cat, that prediction was badly wrong. A common way to quantify this error is the sum of squared residuals: take the difference between the network's prediction and the true value, square it (to ensure it's always positive), and sum it up across every image in the dataset.
The smaller this total, the better the network is performing. The loss is our compass. Training is the process of making it as small as possible.
Backpropagation: Learning from Mistakes
With billions, and in today's large models, trillions of weights and biases to adjust, randomly trying combinations and hoping for the best is not a viable strategy. You don't run a business by hoping to win the lottery. We need a principled method for turning the dials in the right direction. That method is backpropagation.
Using some surprisingly basic calculus (specifically derivatives and the chain rule, stuff you probably learned in high school), backpropagation calculates how each weight and bias influences the loss, and crucially, in which *direction* it needs to move to reduce it. We can't calculate the perfect value for a weight in one step. But we can calculate whether to nudge it up or down to improve the outcome. That's a solvable problem.
Here's how it works in practice. After a forward pass through the network, we start at the output layer and work *backwards*. At each layer, we calculate how much each weight and bias contributed to the loss, then nudge each one a small step in the direction that reduces it. The size of that step is called the learning rate, another design choice. Then we move to the previous layer and repeat, all the way back to the input layer.
This backward journey is why it's called *back*propagation. After one complete pass, every weight and bias has been nudged slightly in the right direction. The model is, very marginally, better than it was before.
Training: The Full Loop
One pass is not enough. After backpropagation, we feed the data through again, measure the loss, and run backpropagation once more. Then again. And again.
In practice, feeding all the data through at once would be computationally overwhelming for large datasets. Instead, the data is split into smaller chunks called batches, say, 32 or 64 images at a time, and the network updates its weights after each batch rather than after seeing every example. A complete pass through the entire dataset is called an epoch. Training typically runs for many epochs, with the loss (hopefully) decreasing a little more with each run.
With each iteration, the weights and biases are tweaked a little further in the right direction. Slowly, the loss decreases. The model's predictions improve. Eventually, after enough passes through the data, the adjustments become vanishingly small, further tweaking no longer improves the outcome. The dials have found their settings.
The model has learned.
Training vs. Inference
It's worth drawing a clear line between two distinct phases in the life of a neural network.
Training is everything described above: feeding data through, measuring loss, running backpropagation, adjusting weights. It is computationally expensive, time-consuming, and happens before the model is released to the world. When you hear that training a large AI model costs millions of dollars, this is what they're referring to.
Inference is what happens when you actually *use* a trained model. You provide an input (a question, an image, a prompt) and the network runs a single forward pass to produce an output. The weights are fixed; nothing is being learned or updated. Inference is fast and cheap compared to training.
When you type a question into your favourite AI tool, you are not training the model. You are running inference on a model that was trained long before you arrived.
A Few Caveats
None of this is guaranteed to work. Sometimes the loss doesn't decrease. Sometimes it improves for a while and then plateaus. The culprit could be the learning rate, the activation function, the loss function, the initial weight values, or the data itself, which may be too noisy, too sparse, or mislabelled.
One particularly common failure mode is overfitting: the model learns the training data *too* well. It memorises the specific examples it was trained on rather than learning the underlying pattern, which means it performs brilliantly on training data but poorly on new, unseen data. It's the difference between a student who truly understands a subject and one who has memorised last year's exam answers - the latter will struggle the moment the questions change. Avoiding overfitting is one of the central challenges in building models that actually generalise to the real world.
A great deal of trial and error goes into getting a neural network to train well. In that sense, building AI models is as much art as it is science. But when it works, it works remarkably well. And this basic framework - neurons, weights, forward passes, loss functions, backpropagation - has remained the engine underneath virtually every modern AI system you've ever encountered.
Now that this machinery is on the table, future posts in this series can build on it freely. When I talk about how language models predict text, why some AI systems generalise better than others, or what made the breakthroughs of the 1980s and 1990s possible, I'll link back here. This is the engine room.