From training models to choosing actions

Back to blog The Learning Studio Blog

Why Optimization Is Everywherein Artificial Intelligence

Train a machine learning model.

Choose the best parameters.

Reduce prediction error.

Allocate resources.

Select the next action.

Maximize reward.

Plan a route.

Tune a neural network.

These may sound like completely different problems.

But underneath many of them lies the same mathematical question:

What is the best choice I can make?

That is the language of optimization.

Once we begin looking for it, optimization appears almost everywhere in Artificial Intelligence.

Machine Learning asks:

Which parameters minimize prediction error?

Reinforcement Learning asks:

Which actions maximize long-term reward?

AI agents ask:

Which next step is most useful?

Planning systems ask:

Which sequence of actions reaches the goal efficiently?

And neural networks repeatedly ask:

How should the parameters change so that the model performs better?

Different problems.

Different algorithms.

But often the same mathematical structure:

\text{Find the choice that makes an objective as good as possible.}

What Does Optimization Actually Mean?

Optimization landscape diagram showing local maxima, local minima, global minimum, and objective values across a surface.

The word can sound intimidating.

But the basic idea is simple.

Suppose we have a function:

\[ f(x) \]

and we want to find the value of \(x\) that makes it as small as possible.

We write:

\min_x f(x)

Perhaps instead we want the largest possible value:

\max_x f(x)

That is optimization.

Imagine a landscape of hills and valleys.

Every point represents a possible solution.

The height represents how good or bad that solution is.

If lower is better:

We want to find the deepest valley.

If higher is better:

We want to find the highest peak.

This simple picture appears again and again in AI.

Only the meaning of the landscape changes.

Machine Learning as an Optimization Problem

Machine learning workflow showing training data, model parameters, predictions, loss function, parameter updates, and before and after optimization.

Imagine we are trying to predict house prices.

Our model makes predictions.

But those predictions are not perfect.

Suppose:

Actual price = 80 lakh

and:

Predicted price = 70 lakh

There is an error.

For many examples, we can combine these errors into what is called a loss function.

One familiar example is Mean Squared Error:

\mathrm{MSE} = \frac{1}{n} \sum_{i=1}^{n} \left(y_i - \hat{y}_i\right)^2

where:

y_i = \text{actual value}

and:

\hat{y}_i = \text{predicted value}

The goal of training becomes:

\min_{\theta} L(\theta)

where:

\theta = \text{model parameters}

and:

L(\theta) = \text{loss produced by those parameters}

In simple language:

Find the model parameters that make the errors as small as possible.

Suddenly, “training a Machine Learning model” becomes an optimization problem.

Why Do We Need a Loss Function?

Suppose the correct value is:

\[ 75 \]

Model A predicts:

\[ 72 \]

while Model B predicts:

\[ 40 \]

Clearly, Model A is much closer.

But a computer needs a precise mathematical way to represent that difference.

That is what the loss function gives us.

For Model A:

\mathrm{Loss}_A = (75 - 72)^2 = 9

For Model B:

\mathrm{Loss}_B = (75 - 40)^2 = 1225

The second prediction receives a much larger penalty.

The loss function therefore gives us something extremely important:

A mathematical definition of “better.”

Without an objective to improve, an optimization algorithm has no direction.

But How Does the Model Find the Minimum?

Imagine standing somewhere on a mountain in thick fog.

You cannot see the entire landscape.

You want to reach the bottom.

What could you do?

You could inspect the slope beneath your feet.

If the ground slopes downward to the left:

Move left.

Check the slope again.

Move again.

Repeat.

That simple intuition leads to one of the most important ideas in Machine Learning:

Gradient Descent

Gradient descent repeatedly moves toward lower values of the loss function.

Instead of examining every possible value, the algorithm uses information about the local slope to decide:

Which direction should I move?

The Mathematics of Gradient Descent

Gradient descent visual explaining slope, learning rate, step-by-step updates, and movement toward a minimum.

Suppose the model has a parameter:

\[ w \]

and a loss function:

\[ L(w) \]

The derivative:

\frac{dL}{dw}

tells us the slope of the loss function.

The update rule becomes:

w_{\text{new}} = w_{\text{old}} - \eta \frac{dL}{dw}

where \(\eta\) is the learning rate.

For a model with many parameters, we use the gradient:

\theta_{t+1} = \theta_t - \eta \nabla L(\theta_t)

The notation may look compact.

But conceptually the process is simple:

Measure the error.
Determine the direction in which the error increases.
Move in the opposite direction.
Measure again.
Repeat.

That idea lies underneath a remarkable amount of modern Machine Learning.

Why Does the Learning Rate Matter?

Learning rate visual comparing too small, appropriate, and too large learning rates during optimization.

Look again at the update rule:

\theta_{t+1} = \theta_t - \eta \nabla L(\theta_t)

The learning rate \(\eta\) controls how large each step is.

If the learning rate is too small:

Training may become extremely slow.

If the learning rate is too large:

The optimizer may repeatedly overshoot the useful region.

Instead of steadily approaching the minimum, it may bounce around it.

This shows us something important.

Optimization is not simply:

“Move downhill.”

It also requires deciding:

How large should each step be?

That small detail can dramatically change whether learning succeeds.

Local Minimum or Global Minimum?

Local minimum versus global minimum chart showing a function with a nearby valley and the best overall valley.

Now imagine that our landscape contains several valleys.

One valley is fairly low.

Another is much deeper.

An optimizer moving downhill may reach the first valley.

Locally, it looks excellent.

But somewhere else in the landscape, a better solution may exist.

This gives us two important ideas.

Local minimum

A point that is lower than nearby alternatives.

Global minimum

The lowest point across the entire search space.

A locally good solution is not necessarily the best solution available across the whole optimization landscape.

For simple problems, we may be able to find the global minimum exactly.

For large Machine Learning systems, the landscape can be enormously complicated.

And that is one reason optimization becomes so interesting.

The system may be searching through an enormous space of possible parameter values without ever examining them all.

Optimization Is Not Only About Training Models

It is tempting to think of optimization only as:

Training a neural network.

But optimization appears before, during and after model training.

We optimize when deciding which features to use.

We optimize hyperparameters.

We optimize how computing resources are allocated.

We optimize schedules.

We optimize routes.

We optimize recommendation strategies.

We optimize actions in Reinforcement Learning.

We optimize decisions made by AI agents.

So optimization is not simply one chapter inside Machine Learning.

It is a recurring mathematical pattern across Artificial Intelligence.

Optimization and Reinforcement Learning

Comparison of supervised learning minimizing loss and reinforcement learning maximizing reward.

In the previous blog, we explored Reinforcement Learning.

There, an agent takes actions and receives rewards.

A common mathematical objective is:

\max_{\pi} \; \mathbb{E}_{\pi} \left[ \sum_{t=0}^{\infty} \gamma^t r_{t+1} \right]

Let us unpack this.

\( \pi \) represents the policy.

\( r_{t+1} \) represents the reward received after an action.

\( \gamma \) is the discount factor.

And \( \mathbb{E}_{\pi} \) represents the expected outcome when following the policy.

The basic idea is:

Find a policy that produces high expected long-term reward.

Now compare the two problems.

In supervised Machine Learning:

\boxed{\text{Minimize Loss}}

In Reinforcement Learning:

\boxed{\text{Maximize Expected Reward}}

Different language.

Different applications.

But the same mathematical family:

Optimization.

Optimization and AI Agents

Now connect optimization to AI agents.

Imagine an AI research agent has several possible actions.

It could:

Search again.
Open another source.
Use another tool.
Perform a calculation.
Ask the user for clarification.
Stop and answer.

Which action should it choose?

A simplified mathematical model could assign a utility to every action:

U(a) = \text{Expected Benefit}(a) - \text{Cost}(a) - \text{Risk}(a)

The system could then choose:

a^* = \operatorname*{arg\,max}_{a} U(a)

This notation means:

Choose the action whose utility is highest.

A real AI agent does not necessarily solve this exact equation every time it acts.

But mathematically, it gives us a useful framework for understanding the problem.

The agent is selecting among alternatives.

That is optimization.

The Cheapest Option Is Not Always the Best

Suppose an AI agent has two possible tools.

Tool A

Cost:

\[ 2 \]

Probability of success:

95\%

Tool B

Cost:

\[ 0.5 \]

Probability of success:

60\%

If we optimize only for cost:

Choose Tool B.

If we optimize only for probability of success:

Choose Tool A.

But perhaps neither objective is sufficient.

We might instead define:

U = \text{Benefit} - \lambda \, \text{Cost}

where \(\lambda\) controls how strongly cost influences the decision.

A large value of \(\lambda\) gives cost greater importance.

A smaller value allows expected benefit to dominate.

This introduces one of the most important ideas in optimization:

Real problems usually involve trade-offs.

AI Rarely Optimizes Only One Thing

Multi-objective optimization visual comparing speed, safety, cost, and quality across different choices.

Imagine designing an autonomous vehicle.

We want it to:

Reach the destination quickly.

But also:

Drive safely.
Use energy efficiently.
Follow traffic rules.
Maintain passenger comfort.

Now suppose these objectives conflict.

The fastest route may consume more energy.

The most energy-efficient route may take longer.

The shortest route may not be the safest.

There may be no single solution that is simultaneously best at everything.

We might therefore create an objective such as:

\[ J = w_1T + w_2E + w_3R \]

where:

T = \text{travel time}, \quad E = \text{energy consumption}, \quad R = \text{risk}

and:

w_1, \; w_2, \; w_3

represent how much importance we give to each objective.

More generally:

J(x) = \sum_{i=1}^{k} w_i f_i(x)

This is one common way to handle multi-objective optimization: convert several goals into one weighted objective.

The weights matter because they quietly express what the system values most.

Constraints Matter Too

Optimization objectives and constraints visual showing possible solutions, objective function, constraints, and feasible best choice.

Optimization does not mean:

Do anything necessary to maximize the score.

Real problems have boundaries.

Mathematically, we might write:

\min_x f(x)

subject to:

g_i(x) \leq 0

for:

i = 1,2,\ldots,m

The objective tells us:

What are we trying to improve?

The constraints tell us:

What rules must the solution obey?

Imagine scheduling staff in a hospital.

We may want to minimize cost.

But we cannot simply schedule fewer and fewer people.

There are constraints:

Minimum staffing levels.
Maximum working hours.
Required expertise.
Emergency coverage.

The cheapest schedule that violates patient-safety requirements is not a useful solution.

This is why:

The mathematically lowest-cost answer is not necessarily the real-world optimum.

The solution must also be feasible.

When the Objective Is Wrong

This connects directly to something we discussed in Reinforcement Learning.

Suppose we tell a system:

Maximize clicks.

Mathematically, we might write:

\max_{\theta} \; \mathrm{Clicks}(\theta)

And suppose the system becomes extremely good at maximizing clicks.

Has it succeeded?

Mathematically:

Perhaps.

But what if the real objective was:

Provide genuinely useful information.

Then the optimizer may be performing perfectly while solving the wrong problem.

This is a profound idea.

An optimization algorithm can be mathematically successful and practically wrong.

The algorithm may faithfully optimize:

\[ f(x) \]

But the more important question is:

Was \(f(x)\) the right thing to optimize?

That is why objective design matters so much.

Accuracy Is Not Always the Only Objective

Consider two Machine Learning models.

Model A

Accuracy:

96\%

Model B

Accuracy:

95\%

Should we automatically choose Model A?

Not necessarily.

Model B might be:

Faster.
Cheaper.
Easier to interpret.
More stable.
Better calibrated.
Easier to deploy.

So instead of optimizing only:

\max \; \text{Accuracy}

the real problem may involve several quantities:

J = w_1(\text{Accuracy}) - w_2(\text{Cost}) - w_3(\text{Latency}) + w_4(\text{Interpretability})

Real AI systems rarely live inside a single metric.

What Are We Really Optimizing?

This may be one of the most useful questions to ask whenever we encounter an AI system.

Instead of asking only:

“Which algorithm is being used?”

ask:

“What is this system trying to optimize?”

Is it optimizing:

Prediction accuracy?
Engagement?
Cost?
Speed?
Reward?
User satisfaction?
Long-term performance?
A combination of several objectives?

And then ask an even more important question:

“Does that mathematical objective actually represent what we want?”

That question connects mathematics directly with system design.

The Mathematics Behind Optimization

Despite the enormous variety of AI systems, several mathematical structures repeatedly appear.

Objective Function

What are we trying to improve?

\[ f(x) \]

Minimization

\min_x f(x)

Maximization

\max_x f(x)

Gradient

The gradient tells us how a function changes with respect to its variables:

\nabla f(x)

For several variables:

\nabla f = \begin{bmatrix} \dfrac{\partial f}{\partial x_1} \\ \dfrac{\partial f}{\partial x_2} \\ \vdots \\ \dfrac{\partial f}{\partial x_n} \end{bmatrix}

Gradient Descent

Move in the opposite direction of the gradient:

x_{t+1} = x_t - \eta \nabla f(x_t)

Constraints

Which solutions are allowed?

g_i(x) \leq 0

Multiple Objectives

How do we balance competing goals?

J(x) = w_1f_1(x) + w_2f_2(x) + \cdots + w_nf_n(x)

Different AI problems may use far more sophisticated mathematics.

But these structures provide an important foundation.

A Simple Way to See It

Summary map of optimization in AI, covering objective functions, minimization, maximization, gradients, constraints, and AI examples.

Consider several familiar AI and Machine Learning problems.

Linear Regression

Find parameters that minimize prediction error:

\min_{\theta} L(\theta)

Neural Networks

Adjust many parameters to reduce a loss:

\theta_{t+1} = \theta_t - \eta \nabla L(\theta_t)

K-Means Clustering

Choose cluster centers that minimize within-cluster distances:

\min_{\mu_1,\ldots,\mu_k} \sum_{i=1}^{n} \min_j \left\|x_i - \mu_j\right\|^2

Reinforcement Learning

Find a policy that maximizes expected cumulative reward:

\max_{\pi} \; \mathbb{E}_{\pi} \left[ \sum_{t=0}^{\infty} \gamma^t r_{t+1} \right]

AI Agents

Choose useful actions:

a^* = \operatorname*{arg\,max}_{a} U(a)

Planning

Find an efficient sequence of actions.

Scheduling

Allocate limited resources effectively.

The applications look completely different.

But mathematically:

They are all searching for better choices.

That is optimization.

Optimization Does Not Mean Perfection

There is another subtle point.

When we say that a system is “optimized,” it does not necessarily mean:

Perfect.

It may mean:

Best found under the chosen objective.
Best within the available constraints.
Best among the alternatives examined.
Good enough given the available computation.

Suppose the entire set of possible solutions is:

\mathcal{X}

Ideally, we might want:

x^* = \operatorname*{arg\,min}_{x \in \mathcal{X}} f(x)

But when \(\mathcal{X}\) is enormous, finding the exact optimum may be extremely difficult.

So practical optimization often searches for:

A very good solution in a very large space.

That distinction matters.

Why Mathematics Matters

This is exactly why mathematical foundations remain important in AI.

If we understand:

Functions

we can understand objectives.

If we understand:

Derivatives

we can understand gradients.

If we understand:

Vectors

we can understand parameter updates.

If we understand:

Probability

we can reason about uncertainty.

And if we understand:

Optimization

we begin to see the common mathematical structure underneath many apparently unrelated AI algorithms.

The tools may change.

The interfaces may change.

The model architectures may change.

But the underlying mathematics remains remarkably useful.

The Bigger Picture

Machine Learning is often presented as a collection of algorithms.

Linear Regression.

Decision Trees.

Neural Networks.

Clustering.

Reinforcement Learning.

Transformers.

AI Agents.

But there is another way to understand Artificial Intelligence.

Instead of asking only:

“Which algorithm is this?”

ask:

“What mathematical problem is being solved?”

Very often, somewhere underneath, we find:

\boxed{\text{Optimization}}

Reduce an error.

Increase a reward.

Balance competing goals.

Search for better parameters.

Choose a better action.

Find a better path.

And that is why optimization appears almost everywhere in Artificial Intelligence.

The Takeaway

Optimization sounds like a specialized branch of mathematics.

But in AI, it is almost a way of thinking.

We begin with:

Many possible choices.

We define:

What “better” means.

We measure:

How good each choice is.

Then we search for:

A better solution.

Again.

And again.

Until the system improves.

In mathematical language, many problems ultimately resemble:

x^* = \operatorname*{arg\,min}_x f(x)

or:

x^* = \operatorname*{arg\,max}_x f(x)

So perhaps one of the most useful questions we can ask about any intelligent system is:

What exactly is it trying to optimize?

Because once we understand the objective:

We understand much more about the behaviour that follows.

Learning with Purpose

At The Learning Studio, my approach to Mathematics, Data Science, Machine Learning and Artificial Intelligence focuses on connecting mathematical ideas to the systems learners actually encounter.

Optimization is a perfect example.

A learner may first see:

\min_x f(x)

and wonder:

“Why do I need this?”

But connect that equation to:

Training a neural network.
Choosing an AI agent’s next action.
Maximizing Reinforcement Learning reward.
Finding the best model.
Balancing cost and performance.

And suddenly the mathematics has a purpose.

The learning journey becomes:

Concept -> Intuition -> Mathematics -> Code -> Experiment -> Question

That is the connection I want learners to see.

Because mathematics in AI is not decoration.

It is part of the machinery underneath the intelligence.

Coming Next

Can an AI Agent Know When It Is Wrong?

Uncertainty, Confidence and the Mathematics of Knowing What You Do Not Know

An AI system gives an answer.

It sounds confident.

But should we trust it?

Suppose a system says:

P(\text{correct}) = 0.95

Does that automatically mean it will actually be correct \(95%\) of the time?

Not necessarily.

What is the difference between:

Confidence

and:

Calibration?

How can mathematics help a system recognize:

“I may be wrong”?

In the next article, we will explore:

Probability.
Confidence.
Calibration.
Uncertainty.
Prediction intervals.
Overconfidence.
Reliability diagrams.
Human escalation.

Because perhaps one of the most important capabilities of an intelligent system is not always knowing the answer.

It is knowing:

When not to be certain.

What Does Optimization Actually Mean?

Machine Learning as an Optimization Problem

Why Do We Need a Loss Function?

But How Does the Model Find the Minimum?

The Mathematics of Gradient Descent

Why Does the Learning Rate Matter?

Local Minimum or Global Minimum?

Optimization Is Not Only About Training Models

Optimization and Reinforcement Learning

Optimization and AI Agents

The Cheapest Option Is Not Always the Best

AI Rarely Optimizes Only One Thing

Constraints Matter Too

When the Objective Is Wrong

Accuracy Is Not Always the Only Objective

What Are We Really Optimizing?

The Mathematics Behind Optimization

A Simple Way to See It

Optimization Does Not Mean Perfection

Why Mathematics Matters

The Bigger Picture

The Takeaway

Learning with Purpose

Coming Next

Explore More Academic Pathways