A machine makes a decision.
Something happens.
The result is good.
Or bad.
The machine changes what it does next.
Then it tries again.
That simple cycle sits behind one of the most fascinating ideas in Artificial Intelligence:
Reinforcement Learning.
The name sounds technical.
The mathematics can certainly become sophisticated.
But the central idea is surprisingly familiar.
Think about how we learn many things ourselves.
A child tries to balance on a bicycle.
Falls.
Adjusts.
Tries again.
A chess player makes a move.
Loses a piece.
Remembers the mistake.
A student tries one method of solving a problem.
It does not work.
They change their approach.
In each case, learning happens through something like:
Action -> Consequence -> Adjustment
Reinforcement Learning, or RL, gives machines a mathematical framework for learning from this kind of experience.
But this immediately raises some interesting questions.
How does a machine know whether an outcome was good?
Should it always repeat what worked before?
When should it try something new?
And what if a small reward today leads to a much larger reward tomorrow?
That is where the mathematics begins.
What Is Reinforcement Learning?
In supervised Machine Learning, we often provide examples together with the correct answers.
For example:
Image -> Cat
Image -> Dog
Email -> Spam
Email -> Not Spam
The model learns from labelled examples.
Reinforcement Learning is different.
Instead of repeatedly telling an agent exactly what the correct answer is, we allow it to interact with an environment.
The agent:
- Observes
- Takes an action
- Receives a reward or penalty
- Learns from the result
- Tries again
At the heart of Reinforcement Learning are four ideas.
State
Where am I now?
Action
What can I do?
Reward
How good or bad was the result?
Update
What should I learn from what just happened?
Then the cycle repeats.
A Robot in a Tiny World
Let us make this concrete.
Imagine a small robot inside a grid.
Its goal is to reach the green square.
It can move:
- Up
- Down
- Left
- Right
Suppose we give it:
\(+10\) for reaching the goal.
\(-10\) for reaching a dangerous square.
And perhaps:
\(-1\) for unnecessary movement.
At the beginning, the robot does not know the best route.
It may wander.
Move toward danger.
Take an unnecessarily long path.
Hit a wall.
Eventually discover the goal.
Then it starts again.
Notice what we have not done.
We have not explicitly told the robot:
Go up, then up, then right, right, right.
Instead, we have given it:
- A goal
- Possible actions
- Feedback
- An opportunity to learn from experience
That distinction is central to Reinforcement Learning.
State, Action and Reward
Now let us introduce a little mathematical notation.
Suppose the agent is currently in state:
\[ s_t \]
It chooses an action:
\[ a_t \]
The environment gives it a reward:
\[ r_{t+1} \]
and it moves to a new state:
\[ s_{t+1} \]
So the interaction can be represented as:
\[ s_t \rightarrow a_t \rightarrow r_{t+1}, s_{t+1} \]
It may look mathematical.
But the idea is extremely simple.
Current situation
Make a decision
Something happens
Learn from it
This structure appears in many decision-making problems.
What Is a Policy?
The agent also needs a strategy.
When it reaches a particular state:
Which action should it choose?
That strategy is called a:
Policy
and is often represented by:
\[ \pi(a \mid s) \]
This can be interpreted as:
Given state \(s\), what action \(a\) should I choose?
For our maze robot, a policy might eventually learn:
If I am here:
Move right.
If I am there:
Move up.
If I am near the dangerous square:
Avoid moving down.
Initially, the policy may be poor.
With experience, it can improve.
What Exactly Is a Reward?
A reward is a numerical signal.
For example:
| Outcome | Reward |
|---|---|
| Reach goal | \(+10\) |
| Dangerous state | \(-10\) |
| Unnecessary step | \(-1\) |
The numbers themselves are not intelligence.
They simply tell the agent:
“This outcome was better.”
or:
“This outcome was worse.”
But here is where things become interesting.
Imagine we tell an autonomous delivery system:
“Reach the destination as quickly as possible.”
If speed is the only thing being rewarded, what might happen?
Perhaps it chooses unsafe routes.
Now imagine telling an AI system:
“Complete as many tasks as possible.”
It might finish many tasks quickly.
But perhaps the quality becomes poor.
So we arrive at one of the deepest lessons in Reinforcement Learning:
The agent learns to optimize the reward we define, not necessarily the goal we intended.
That means designing the reward can sometimes be harder than designing the algorithm.
Learning Through Trial and Error
At the beginning of training, an RL agent may be terrible at the task.
That is expected.
Consider our robot.
Its experience might look something like:
| Episode | What happens |
|---|---|
| 1 | Wanders randomly. |
| 2 | Finds the dangerous square. |
| 4 | Discovers part of a useful route. |
| 9 | Reaches the goal. |
| 15 | Begins reaching it more consistently. |
| 20 | Uses a much better path. |
If we plotted the reward received during training, the line would not need to improve perfectly at every step.
Learning may look like:
- Improvement
- Mistake
- Improvement
- Exploration
- Another mistake
- Better strategy
That is normal.
A learning system can become better even when individual experiences are imperfect.
Should the Agent Always Use What Already Works?
Suppose the robot discovers a route that reaches the goal.
Should it use that route forever?
Maybe.
But what if an even shorter route exists?
This creates one of the most famous problems in Reinforcement Learning:
Exploration vs Exploitation
Exploitation
Use the best strategy currently known.
Exploration
Try something different in case a better strategy exists.
Humans face this problem constantly.
Imagine you have a favourite restaurant.
You know the food is good.
Returning to it is:
Exploitation
Trying a new restaurant is:
Exploration
The new restaurant might be terrible.
But it might also become your new favourite.
AI agents face the same trade-off.
If an agent exploits too much:
It may never discover something better.
If it explores too much:
It wastes time trying poor alternatives.
One simple strategy is known as \(\varepsilon\)-greedy.
Most of the time:
Choose the best-known action.
Occasionally:
Try something else.
Mathematically:
\[ P(\text{explore}) = \varepsilon \]
and:
\[ P(\text{exploit}) = 1 - \varepsilon \]
Suppose:
\[ \varepsilon = 0.10 \]
Then approximately:
\(90%\) of the time the agent uses its best-known action.
\(10%\) of the time it explores.
This is a beautiful mathematical version of a very human question:
Should I stick with what works, or try something new?
The Best Reward Now May Not Be the Best Decision
Now imagine the agent has two possible actions.
Action A
Immediate reward:
\(8\)
Action B
Immediate reward:
\(4\)
Which is better?
At first:
Action A.
But suppose Action A leads to a poor future state.
And Action B leads to a state where the agent can later gain another:
\(10\)
Now things change.
The agent must distinguish between:
What feels best immediately
and:
What produces the best result eventually.
This is one of the central ideas in Reinforcement Learning.
How Much Should the Future Matter?
To model future consequences, RL uses something called the:
Discount Factor
usually written:
\[ \gamma \]
or \(\gamma\).
Typically:
\[ 0 \leq \gamma \leq 1 \]
If \(\gamma\) is small:
The agent cares mainly about immediate rewards.
If \(\gamma\) is close to \(1\):
Future rewards matter much more.
Consider a future reward of:
\(10\)
With:
\[ \gamma = 0.5 \]
its value falls rapidly as that reward gets further away.
With:
\[ \gamma = 0.9 \]
the future retains much more importance.
So \(\gamma\) is really answering a fascinating question:
How much should the future influence what I do today?
An RL agent may therefore care about more than the next reward.
It may care about the return: the total discounted reward expected in the future.
A common expression is:
\[ G_t = r_{t+1} + \gamma r_{t+2} + \gamma^2 r_{t+3} + \cdots \]
Suppose the next three rewards are:
\(4, 3, 5\)
and:
\[ \gamma = 0.9 \]
Then:
\[ G = 4 + 0.9(3) + 0.9^2(5) \]
\[ G = 4 + 2.7 + 4.05 \]
\[ G = 10.75 \]
So the agent is not simply asking:
“What will happen next?”
It is asking:
“What future does this action create?”
What Is a \(Q\)-Value?
Now suppose we want to estimate how good a particular action is in a particular state.
We can represent this using:
\[ Q(s,a) \]
A \(Q\)-value can be interpreted roughly as:
How valuable is action \(a\) when I am in state \(s\)?
Imagine our robot stands near the goal.
It has two choices.
Move right
\[ Q(s,\text{right}) = 9 \]
Move left
\[ Q(s,\text{left}) = 2 \]
The agent would normally prefer:
Right.
But where do those \(Q\)-values come from?
The agent must learn them.
\(Q\)-Learning Without the Fear
One famous Reinforcement Learning algorithm is:
\(Q\)-learning
The update rule is:
\[ Q_{\text{new}}(s,a) = Q(s,a) + \alpha \left[r + \gamma \max_{a’} Q(s’,a’) - Q(s,a)\right] \]
That looks intimidating.
But its meaning is far simpler than the formula suggests.
The agent asks:
What did I previously think this action was worth?
Then:
What reward did I actually receive?
Then:
How promising is the state I reached?
And finally:
How much should I update my old belief?
Let us use actual numbers.
Suppose:
\[ Q(s,a) = 4 \]
Immediate reward:
\[ r = 3 \]
Best future \(Q\)-value:
\(8\)
Learning rate:
\[ \alpha = 0.5 \]
Discount factor:
\[ \gamma = 0.9 \]
First calculate the target:
\[ 3 + 0.9(8) = 10.2 \]
Then:
\[ Q_{\text{new}} = 4 + 0.5(10.2 - 4) \]
\[ Q_{\text{new}} = 4 + 3.1 \]
\[ Q_{\text{new}} = 7.1 \]
The agent originally believed the action was worth:
\(4\).
After experiencing a better-than-expected result, it updates that estimate to:
\(7.1\).
That is learning.
What Does the Learning Rate Mean?
The previous equation included:
\[ \alpha \]
This is the learning rate.
It controls how strongly a new experience changes what the agent already believes.
If \(\alpha\) is very small:
The agent changes slowly.
If \(\alpha\) is large:
Recent experiences influence it strongly.
There is a simple human analogy.
Imagine you have eaten at the same restaurant twenty times.
Nineteen experiences were excellent.
One was poor.
Should that one poor dinner completely change your opinion?
Probably not.
But if the next ten visits are also poor:
Your belief should eventually change.
The learning rate controls how quickly new evidence modifies previous experience.
The Problem of Delayed Consequences
There is another challenge.
Sometimes the action that causes success or failure happened much earlier.
Imagine a chess game with 60 moves.
You lose.
Which move caused the loss?
Move 60?
Move 45?
Move 20?
Perhaps the real mistake happened on:
Move 12.
This is known as the:
Credit Assignment Problem
The final outcome may be clear.
But determining which earlier actions deserve credit or blame can be difficult.
Humans face the same problem.
A successful exam result may be influenced by months of study.
A failed project may trace back to an early design decision.
Learning from consequences requires connecting:
Outcome
back to:
Earlier decisions.
What If We Reward the Wrong Thing?
This is where Reinforcement Learning becomes much more than an algorithm.
Imagine rewarding an AI agent for:
Completing the task as quickly as possible.
Perhaps it starts skipping verification.
Now reward it for:
Producing the longest possible answer.
Perhaps it writes unnecessarily long responses.
Reward:
Maximum engagement.
Perhaps engagement rises.
But did the system become more useful?
Not necessarily.
The mathematics may be doing exactly what we asked.
The problem is:
We asked for the wrong thing.
That is why a reward function should never be thought of casually.
We need to ask:
- What behaviour might this reward accidentally encourage?
- What shortcuts might the agent discover?
- What constraints should also apply?
- What happens in unusual situations?
Good reward design requires:
Mathematics + domain knowledge + evaluation + human judgment.
Reinforcement Learning and AI Agents
This connects directly with the AI-agent series we have been exploring.
We previously asked:
- How does an AI agent choose what to do?
- How does it plan?
- How does it know when to stop?
- What happens when several agents disagree?
Reinforcement Learning gives us one mathematical framework for learning which sequences of actions tend to produce better long-term outcomes.
But there is an important distinction.
Not every modern AI agent is using Reinforcement Learning while it operates.
Many AI agents today combine:
- Language models
- Tools
- Search
- Memory
- Rules
- Planning
- Workflow logic
Reinforcement Learning is a framework for learning sequential decision-making.
It should not be confused with every type of agentic AI.
Understanding that distinction helps us replace:
“The AI somehow learned.”
with better questions:
- What was the state?
- What actions were available?
- What reward was used?
- What behaviour was optimized?
- What feedback changed the policy?
Those are much more useful questions.
Where Can Reinforcement Learning Be Useful?
RL is especially interesting when decisions happen one after another and each action changes what happens next.
Examples can include:
Robotics
How should the robot move?
Games
Which move improves the chance of winning?
Resource Allocation
Where should limited resources be assigned?
Scheduling
Which task should happen next?
Control Systems
How should a system respond as conditions change?
Sequential Recommendations
Which recommendation should come next based on previous responses?
Although these applications look very different, they share a mathematical structure:
Observe
Decide
Act
Receive feedback
Adjust
Repeat
Does the Machine Actually Want the Reward?
No.
Suppose an agent receives:
\(+10\)
The machine does not feel happy.
The reward is simply a numerical signal used by the learning process.
Likewise:
\(-10\)
does not mean the machine feels disappointment.
It means the algorithm treats that outcome as less desirable.
This distinction matters because we often use human language when describing AI:
“The agent wants…”
“The model likes…”
“The AI prefers…”
These can be convenient ways of speaking.
But mathematically, what may actually be happening is:
The system assigns different expected values to different actions.
That is very different from human motivation.
The Mathematics Behind Reinforcement Learning
Once we strip away the jargon, a surprisingly small set of ideas appears repeatedly.
State
Where am I?
\[ s \]
Action
What can I do?
\[ a \]
Reward
What happened?
\[ r \]
Policy
What should I do in this state?
\[ \pi(a \mid s) \]
Return
What is the long-term reward?
\[ G_t = \sum_{k=0}^{\infty} \gamma^k r_{t+k+1} \]
\(Q\)-value
How useful is this state-action pair?
\[ Q(s,a) \]
Learning rate
How much should this new experience change what I already believe?
\[ \alpha \]
Discount factor
How much should future rewards matter?
\[ \gamma \]
Exploration rate
How often should I try something new?
\[ \varepsilon \]
The symbols may look unfamiliar.
The questions behind them are not.
They are questions about:
- Choices
- Consequences
- Uncertainty
- Experience
- Learning
- The future
So what is Reinforcement Learning?
At its simplest:
An agent tries something, observes what happens, and uses the consequence to make better future decisions.
But behind that simple sentence lies a fascinating set of mathematical trade-offs.
- Immediate reward vs future reward
- Exploration vs exploitation
- Old knowledge vs new experience
- Speed vs safety
- Reward maximization vs the real objective
Reinforcement Learning is therefore not simply:
“Reward the good behaviour and punish the bad.”
It is about learning:
Which decisions lead to better outcomes over time.
The Bigger Lesson
Perhaps the most important lesson is not only about Reinforcement Learning.
It is about how we learn AI.
New technologies often arrive wrapped in terminology.
Agents.
Policies.
\(Q\)-values.
Rewards.
Exploration.
Bellman equations.
At first, the vocabulary can make the subject seem more complicated than it really is.
But when we ask:
- What is the situation?
- What choices are available?
- What happens after the choice?
- How do we measure success?
- How much should the future matter?
the mathematics begins to have a purpose.
And once the mathematics has a purpose:
The algorithm becomes much easier to understand.
Learning with Purpose
At The Learning Studio, my approach to Mathematics, Data Science, Machine Learning and Artificial Intelligence is based on building those connections.
A learner does not need to begin Reinforcement Learning by memorizing complicated equations.
A better sequence is:
Situation -> Decision -> Consequence -> Intuition -> Mathematics -> Code -> Experiment
Start with the problem.
Understand the behaviour.
Introduce the mathematics.
Then write the code.
Because AI tools will continue to change.
Algorithms will evolve.
Frameworks will come and go.
But the ability to reason about:
decisions, uncertainty, rewards, trade-offs and consequences
will remain valuable.
Coming Next
Why Optimization Is Everywhere in Artificial Intelligence
From Training Models to Choosing Actions
Machine Learning repeatedly asks us to:
- Minimize loss
- Maximize reward
- Find the best parameters
- Choose the best action
- Balance competing objectives
Why?
Because underneath a surprising amount of AI lies one mathematical idea:
Optimization
In the next article, we will explore how an expression as simple as:
\[ \min_x f(x) \]
connects to:
- Training neural networks
- Gradient descent
- Model fitting
- Reinforcement Learning
- AI agents
- Resource allocation
- Decision-making
Because perhaps one of the best ways to understand modern AI is to ask:
“What exactly is this system trying to optimize?”