Imagine a laboratory with enough time and material to run only ten experiments.
The AI has 10,000 possibilities.
One candidate already looks excellent.
Nine thousand others look mediocre.
A few are strange, poorly understood, and potentially transformative.
What should the AI do?
Test the best-looking candidate ten times?
Try ten different candidates?
Spend most of the budget on the safe option and reserve one experiment for something unusual?
This is one of the most important decisions in autonomous science.
It is the tension between exploitation and exploration.
Prefer to watch? Continue with the video below, or keep reading.
The Temptation to Choose the Best
Suppose an AI model predicts the performance of a candidate material.
Candidate A has an expected score of 95.
Candidate B has 92.
Candidate C has 70.
The obvious choice appears to be A.
That is exploitation:
Use what we already know to choose the option that currently looks best.
If the goal is simply to maximize immediate performance, exploitation makes sense.
But scientific research has another goal.
We also want to learn.
And the experiment that gives the best immediate result is not always the experiment that teaches us the most.
The Strange Candidate
Now add uncertainty.
Candidate A has predicted performance
\[ 95 \pm 1. \]
Candidate B has
\[ 92 \pm 2. \]
Candidate C has
\[ 70 \pm 30. \]
Candidate C looks worse on average, but we know very little about it.
Its true value might be 40.
Or 100.
If we never test uncertain regions, we may miss the best discovery.
This is exploration:
Spend some resources learning about possibilities that are uncertain but potentially valuable.
Science lives in the balance between these two instincts.
Scientists Already Do This
Human researchers rarely use the words exploitation and exploration in ordinary laboratory conversation, but the distinction is familiar.
A scientist may spend most of a grant improving a known method.
That is exploitation.
Then she reserves time for a risky experiment because the result could open a new direction.
That is exploration.
The difficulty is that human intuition does not scale well when the candidate space contains millions of possibilities.
AI can help make this tradeoff explicit.
The Value of an Experiment Is Not Just Its Outcome
Suppose two experiments cost the same.
Experiment 1 is likely to confirm something we already strongly believe.
Experiment 2 has no clear expected winner, but either possible result would sharply distinguish between competing models.
Which is more valuable?
If the goal is knowledge, Experiment 2 may be better.
This leads to the idea of information gain.
An experiment has value not only because of the result it might produce, but because of how much that result could change our understanding.
We can think loosely of the goal as choosing
\[ x^* = \arg\max_x \left[ \text{expected scientific value of measuring }x \right]. \]
The exact definition of “scientific value” depends on the problem.
It might mean reducing uncertainty.
Improving a prediction.
Distinguishing between mechanisms.
Finding an optimum.
Or discovering a completely new regime.
Active Learning: Ask for the Most Useful Data
In ordinary machine learning, we often begin with a fixed dataset.
In active learning, the learner helps decide which data it wants next.
Imagine training a model to classify samples.
Instead of labeling every possible sample, the model asks for labels only where it is uncertain or where the answer would be particularly informative.
The same idea can be transferred to experiments.
The laboratory does not simply collect more data.
It collects the next most useful data.
This is why active learning is so important for self-driving laboratories.
Physical experiments are expensive. We want each one to reduce uncertainty as efficiently as possible.
Bayesian Optimization: Search with Uncertainty
Another common approach is Bayesian optimization.
Suppose an experiment has input settings $x$ and produces a performance value $f(x)$.
We do not know $f$ everywhere.
So we build a surrogate model—often a Gaussian process—that predicts both a likely value and an uncertainty.
Then an acquisition function chooses where to experiment next.
Two standard examples make the tradeoff concrete:
- Expected Improvement (EI) asks how much better a candidate is expected to be than the best result found so far.
- Upper Confidence Bound (UCB) combines predicted performance with uncertainty, deliberately rewarding exploration of regions that may be better than the current model expects.
Different acquisition functions therefore encode different attitudes toward risk, uncertainty, and the value of new information.
The resulting strategy can be written conceptually as
\[ x_{\text{next}} = \arg\max_x a(x), \]
where $a(x)$ balances what looks good against what remains unknown.
This is a mathematical version of scientific curiosity.
Search Trees and Scientific Paths
Some problems are not single-step optimization.
One experiment changes what experiments become possible later.
Consider synthesizing a complex material.
Step 1 determines what intermediates can be created.
Step 2 depends on Step 1.
The research process becomes a tree.
At every branch, the system must decide which path deserves more resources.
This resembles planning and reinforcement learning.
A branch that looks promising may deserve deeper exploration.
But if all resources flow into one branch too early, the system can become trapped in a local optimum.
Scientific search is therefore not only about choosing the best experiment.
It is about managing a portfolio of possible futures.
Failure Can Be Highly Informative
Human researchers often dislike failed experiments.
An autonomous system should learn to value some failures.
Suppose a model predicts that a reaction should work under a broad range of temperatures.
A carefully chosen experiment finds a temperature at which it suddenly fails.
That failure may expose a missing mechanism.
A successful experiment might have taught us less.
This is one reason scientific optimization differs from ordinary engineering optimization.
The highest-performing point is not always the most scientifically interesting point.
An anomaly can be more valuable than a success.
The Objective Function Is the Hidden Scientist
Every autonomous research system contains an objective, whether explicit or implicit.
Maximize yield.
Minimize cost.
Reduce uncertainty.
Find a material above a threshold.
Fit the observed data.
The choice matters enormously.
If we tell the AI to maximize a narrow metric, it may become extremely good at that metric while ignoring everything else.
This is the scientific version of the familiar problem of optimizing the wrong target.
So before asking
How should AI choose the next experiment?
we must ask
What do we actually want the experiment to accomplish?
That question is often human.
Exploration Is Not Randomness
Exploration does not mean trying arbitrary things.
Good exploration is structured.
The system may target:
- regions of high uncertainty,
- disagreements between models,
- unexplored mechanisms,
- points near a predicted phase transition,
- experiments that discriminate between hypotheses.
The goal is not surprise for its own sake.
It is useful surprise.
Human Scientific Taste Still Matters
An algorithm can calculate uncertainty.
It can estimate expected information gain.
It can rank candidate experiments.
But scientists often notice something harder to encode:
“This result feels strange.”
“This assumption has never really been tested.”
“Everyone is optimizing the same region. Maybe the interesting physics is somewhere else.”
This is scientific taste.
It is partly knowledge, partly intuition, and partly experience.
A mature AI4S system may eventually reproduce some of this behavior.
For now, the best systems may combine machine-scale search with human judgment about what deserves attention.
The Next Experiment Is the Real Unit of Progress
Scientific AI is often evaluated by the quality of its predictions.
But in a closed-loop laboratory, another metric may matter more:
Does the system choose good next experiments?
A model can be imperfect and still be an excellent experimental guide if it knows where its uncertainty matters.
Conversely, a highly accurate model can be scientifically unproductive if it always recommends safe, unsurprising tests.
The deeper goal is not prediction alone.
It is efficient learning from interaction with the world.
And that brings us to a difficult question.
If an AI predicts the outcome of an experiment correctly, has it actually discovered anything?
Next in the series: Prediction Is Not Discovery
0 comments