Artificial Intelligence and Statistics: The Mathematical Shortcut That Makes Simulations Up to 100 Times Faster
Every day, thousands of researchers around the world use algorithms that simulate millions of possible scenarios to answer very concrete questions: How effective is a new drug? How will an epidemic spread? What is the risk of a financial investment? How reliable is an artificial intelligence model?
Behind these analyses lies a family of statistical methods known as Markov Chain Monte Carlo (MCMC). These are essential tools of Bayesian statistics and modern machine learning because they allow researchers to explore problems with hundreds or thousands of parameters, where an exact calculation would be practically impossible. However, there is a limitation that has plagued these algorithms since their inception in the 1950s: they were designed to proceed one step at a time. Each new calculation depends on the previous one. It’s like reading a novel: you can’t understand chapter ten without having read chapter nine. Today, however, computers have changed. Servers, clusters, and GPUs have dozens or hundreds of processors capable of working simultaneously. The paradox is that many of the most important statistical algorithms manage to utilize only a small fraction of them.
This is precisely the problem addressed by Giacomo Zanella and Sebastiano Grazzi (both from the Department of Decision Sciences and the BIDSA research center at Bocconi) in their study “Parallel computations for Metropolis Markov chains with Picard maps,” recently published in Biometrika.
Their proposal paves the way for a new generation of algorithms capable of utilizing parallel computing much more efficiently, drastically reducing the time required to run complex statistical simulations.
Why do these simulations take so long?
MCMC algorithms work by constructing a long sequence of possible solutions. Each step uses the information produced by the previous one, progressively improving our understanding of the problem.
This dependency makes it difficult to distribute the work across multiple processors. The simplest way to take advantage of a powerful computer is to run many independent simulations, but this strategy does not speed up the most computationally expensive part of the process: the time required for each simulation to converge to a reliable result.
“The simplest approach to parallelizing computations is to run multiple independent chains. This is straightforward to implement in parallel, but does not reduce the convergence period.”
In other words, you get more simulations in the same amount of time, but each one still takes the same amount of time to “learn” the probability distribution you want to study.
The idea: anticipating work instead of waiting
The innovation introduced by the study consists of a complete shift in perspective. Instead of waiting for each step to be completed before starting the next one, the new algorithm attempts to anticipate part of the simulation. Multiple processors work simultaneously to make predictions about the next steps; the algorithm then quickly verifies which predictions were correct and corrects only the incorrect ones. The process is repeated several times until the result matches what the traditional algorithm would have produced.
Behind this mechanism lies a mathematical construct called a Picard map, but the fundamental message is much more intuitive: instead of performing millions of operations strictly one after another, it becomes possible to perform many of them in parallel without losing accuracy.
A theoretical breakthrough that changes the rules of the game
The work not only proposes an elegant idea: it also demonstrates, using rigorous mathematical tools, that this strategy yields a speedup that is practically proportional to the number of processors used, at least for a broad class of very common statistical problems.
For some high-dimensional simulations, this means reducing computation times by a factor equal to the square root of the number of parameters in the problem. In approximate versions of the algorithm, experiments show even greater speedups—more than a hundred times faster than sequential implementations.
The authors also highlight a particularly significant result:
“To the best of our knowledge, this is the first parallel zeroth-order MCMC scheme with provably linear speed-up in the canonical log-concave setup.”
In other words, it is the first method in this family for which it can be mathematically proven that adding processors yields a performance gain that is practically proportional.
When there are no mathematical shortcuts
One of the most interesting aspects of the study is that the proposed algorithms work even when the model to be analyzed is a true “black box.” In many applications, in fact, it is not possible to easily calculate the mathematical derivatives of the model, because the software is proprietary, uses very complex numerical simulators, or incorporates code developed by experts from different disciplines.
In these cases, the fastest traditional algorithms cannot be used. The method developed at Bocconi, on the other hand, requires only the ability to evaluate the model at various points in the simulation, making it applicable to a very wide range of real-world problems.
From epidemiology to precision medicine
To verify the effectiveness of the approach, the authors tested it on a wide variety of problems. The simulations include high-dimensional statistical models, an epidemiological model used to study the spread of infectious diseases, and a real-world precision medicine application dedicated to personalizing cancer therapies.
This last example is particularly significant. Each model evaluation requires the numerical solution of complex differential equations and takes about a quarter of a second. Under these conditions, the new algorithm achieved a reduction in total execution time of more than two and a half times compared to the traditional version—a result that could translate into much greater savings on even more complex models.
Research looking toward the future of scientific computing
Grazzi and Zanella’s contribution goes beyond the improvement of a specific algorithm. More and more scientific models are run on computers equipped with tens or hundreds of processors, while many statistical tools continue to be designed as if they were meant to run on a single processing unit.
The new approach helps bridge this gap. When each simulation takes a long time—as is the case in precision medicine, epidemiology, computational biology, or many applications of artificial intelligence—truly leveraging parallel computing can significantly reduce analysis times, allowing researchers to explore increasingly complex models more quickly.
The authors themselves point to several avenues for further development, with the goal of making these algorithms even more efficient. But the main result is already clear: even a well-established technique that has been in use for over seventy years can be reimagined to adapt to 21st-century hardware. And it is precisely through innovations like this—often invisible to the general public—that many of the future advances in scientific research will be made.
