Contacts

When a Wrong Model Gets the Right Answer

, by Andrea Costa
Scientists routinely use simplified models for reality because the real calculations are impossible. New research investigates when that gamble pays off and when it doesn’t

Suppose you wanted to understand how the Universe evolved after the Big Bang. In principle, you would build a mathematical model that captures every relevant physical process, every interaction and every source of uncertainty. In practice, that model would be so complicated that running it might take an impossibly long time, or simply exceed today’s computing power.

Facing this problem, scientists do something more pragmatic. They knowingly use the wrong model. Not a completely wrong one, of course, but a simpler one. A version of reality that ignores some of its complexity in exchange for the ability to produce an answer.

For decades, this has been one of science’s open secrets. Researchers in fields ranging from astronomy to medical imaging and climate science routinely rely on simplified statistical models because the exact ones are simply too difficult to use. But if the model is wrong, why should anyone trust the answer?

A new study published in Bernoulli tackles precisely that question in context of physical systems governed corrupted with measurement errors. Botond Szabó (Department of Decision Sciences and BIDSA, Bocconi University), together with Geerten Koers and Aad van der Vaart of Delft University of Technology, has developed a mathematical framework that explains when a class of such simplified models remain trustworthy—and, just as importantly, what their limitations are.

Being wrong isn’t the same as being misleading

At first glance, the conclusion seems almost paradoxical: a statistical model can misrepresent reality and still identify the correct answer. How is that possible?

Imagine trying to sketch the route from A to B. A motorway map leaves out thousands of details: buildings, rivers, side streets and traffic lights. In that sense, it is "wrong." Yet if your only goal is to drive from one city to the other, those missing details hardly matter.

A scientific model works in much the same way. Researchers often don’t need to reproduce every aspect of reality. They only need to preserve the information that matters for the question they are trying to answer. The challenge is knowing which simplifications are harmless and which change the answer itself. That is the problem the paper studies.

Rather than analyzing one specific approximation, the authors develop a general mathematical framework that tells scientists when a broad class of simplified statistical models still leads to reliable conclusions, even though the models themselves are imperfect.

The result comes with an important twist. The simplified model often succeeds in finding the right answer. What it may not get exactly right is how certain scientists should feel about that answer. In other words, the biggest danger is not necessarily reaching the wrong conclusion. It is becoming either too confident—or not confident enough—that the conclusion is correct. That distinction lies at the heart of the paper.

A puzzle inspired by the universe

The research began with a real scientific problem. Cosmologists study the faint radiation left over from the Big Bang—the cosmic microwave background—to estimate fundamental properties of the Universe, including the amount of dark matter, the amount of dark energy and the Hubble constant, which measures how fast the Universe is expanding.

Those observations are linked to the underlying physics through extraordinarily complicated chains of processes. Analyzing them exactly is often unrealistic, so researchers rely on simplified statistical models instead. Sometimes, however, different simplifications produce different estimates.

That raises an awkward question. Are scientists seeing evidence of new physics—or are they simply seeing the consequences of making different statistical approximations? As the authors write:

"Understanding the theoretical properties of misspecified hierarchical approaches is crucial to determine whether these non-overlapping confidence intervals result from using different models or statistical error."

The paper does not settle the debate over the expansion of the Universe. What it does provide is a way of distinguishing disagreements that come from nature from disagreements that come from the mathematics used to look at it.

Proving the shortcut works

The authors’ achievement is not that they found a better approximation. Instead, they proved something more general. They developed a mathematical theorem that applies to an entire family of complex statistical models—the kind used whenever observations are generated through several hidden layers, from latent biological processes to physical equations describing quantum systems or the evolution of heat.

Once the framework was in place, they tested it on increasingly demanding examples. They began with a deliberately simple mathematical problem before tackling the Schrödinger equation, one of the cornerstones of quantum mechanics, and then moved on to more general equations used to describe evolving physical systems. These are exactly the kinds of problems where exact calculations rapidly become impractical.

The researchers also checked whether their theory held up in practice. Using simulated datasets ranging from 50 to 1,250 observations, they found that the simplified methods behaved just as the mathematics predicted. As more data became available, the estimates rapidly converged towards the correct answer.

Perhaps just as importantly, the computational savings were dramatic. In one example, the simplified approach could be computed in a fraction of the time required by the full statistical model while avoiding the numerical difficulties that often make exact calculations prohibitively expensive.

Why this changes more than statistics

The paper is, at its heart, a work of mathematical statistics, but its message reaches much further. Science is increasingly shaped by a simple reality: our models are becoming more detailed faster than our ability to compute them exactly. In many fields, approximation is no longer a convenience—it is a necessity. That makes trust one of science’s most valuable resources.

Researchers need to know not only whether a shortcut is faster, but whether it changes the scientific conclusion. This study offers a reassuring answer. Sometimes, the smartest scientific strategy is not to build a perfect model of the world. It is to build a simpler one and to understand, with mathematical precision, why that simplification still captures the truth.

Botond Szabo

BOTOND TIBOR SZABO

Bocconi University
Department of Decision Sciences