A new study from MIT researchers challenges a long-standing assumption in artificial intelligence, demonstrating that general-purpose training algorithms can outperform specialized gameaaaa-theoretic methods in complex, imperfect-information games. The findings, presented at the International Conference on Learning Representations in Rio de Janeiro, upend conventional wisdom about how neural networks should be trained for strategic decision-making and carry implications far beyond the poker table.
Imperfect-information games — situations where players must act without full knowledge of their opponent’s state — are everywhere. A poker hand, a bidding war for a home, a military operation, or a corporate negotiation all involve decisions made under partial information. For years, the dominant view in AI research held that algorithms built specifically on game theory principles would naturally outperform general-purpose approaches in these settings. The MIT-led study suggests that this assumption needs rethinking.
What Policy Gradient Methods Revealed About Strategic Play
The research compares two broad families of algorithms: specialized game-theoretic methods designed for multi-agent competition, and policy gradient methods, a general class of algorithms introduced in the 1990s for training neural networks to make sequential decisions toward a goal. Policy gradient methods work by moving in small, incremental steps along a gradient — a path of greatest improvement — while continuously adjusting course. In single-agent settings, these methods have proven highly effective, but their application to multi-agent games has long been viewed with skepticism.
“It had been pretty much taken for granted that specialized game-theoretic algorithms were the right approach for this setting,” says Samuel Sokota, a co-author from Carnegie Mellon University. “Our study showed that policy gradient methods can work better than these specialized algorithms, and that the specialized algorithms may not work as well as people thought — which raises an interesting sociological question about why this went unnoticed for so long.”
A Benchmark Designed for Fair Comparison
Rather than proposing yet another new algorithm, the research team focused on building a fair and rigorous evaluation framework. “We’re not proposing a new algorithm that can beat out other algorithms,” notes Max Rudolph of the University of Texas at Austin. “We’re proposing a benchmark that can assess these algorithms.”
The benchmark computes a player’s performance using a measure called exploitability, which quantifies how well a strategy performs against a worst-case adversary. In this framework, a score of zero represents perfect play, while higher scores indicate increasing vulnerability to exploitation. The researchers scaled this metric to handle games of enormous complexity — some with as many as 30 billion distinct states, where a state encompasses not just board positions but the full history of every move and misstep.
“It’s like looking into a dark room that’s filled with objects you can’t see,” says Sobhan Mohammadpour, a PhD student at MIT. “Somehow, you need to figure out where these objects are and exactly how they got there.” Previous work using exploitability typically handled games roughly 100,000 times smaller than those analyzed in this study.
Testing Across Five Imperfect-Information Games
The team tested their approach across five games: two versions of Phantom Tic-Tac-Toe, in which players cannot see their opponent’s moves; two imperfect-information variants of the board game Hex; and Liar’s Dice, a game of deception and bluffing. In every case, neural networks trained with policy gradient methods achieved better — lower — exploitability scores than networks trained on specialized game-theoretic algorithms. In direct head-to-head competitions, the policy gradient-trained networks consistently defeated their game theory-trained opponents.
“Those results were reassuring,” Rudolph says, “because they give us more confidence in our benchmarking approach.”
Why This Matters Beyond Recreational Games
The implications of this work extend well beyond board games and card games. Gabriele Farina, an assistant professor at MIT and a principal investigator at LIDS, emphasizes that the term “game” applies broadly to any multi-agent strategic interaction. “The lessons we learn from this research are by no means limited to recreational games,” he says.
Eugene Vinitsky of New York University agrees. “Hidden information is a very important property of the world. It pervades a range of things — including military operations, trading scenarios, and negotiations — all of which are carried out under conditions of hidden information. The idea that we can improve on these games suggests that we can also do better in these other settings as well.”
Ian Gemp, a computer scientist and game theory expert at Google DeepMind who was not involved in the study, describes the findings as encouraging. “This work serves as a compelling reminder that modernizing classical tools remains a highly productive path for solving complex strategic problems.”
An Open and Accessible Tool for the Research Community
The MIT team has made their benchmarking software freely available and designed it for ease of use. “You don’t need a supercomputer,” Mohammadpour says. “You can run it on an ordinary laptop. And all you have to do is add a single line of code to a commonly used collection of benchmarking software called OpenSpiel.”
This accessibility lowers the barrier for researchers and practitioners who want to evaluate their own algorithms against a standardized, rigorous framework. The benchmark’s ability to handle games with tens of billions of states on consumer hardware represents a significant engineering achievement in its own right.
What This Means for AI Practitioners
For developers and researchers working on multi-agent systems, the study offers a practical lesson: before reaching for a specialized, domain-specific algorithm, it is worth testing whether a well-tuned general-purpose method might perform just as well — or better. The findings suggest that the gap between general and specialized approaches may be narrower than the field has assumed, and that rigorous benchmarking is essential for making informed algorithmic choices.
The team’s benchmark is available now, integrated into OpenSpiel, and ready for use on standard hardware. For anyone training neural networks to operate in environments with hidden information — whether in games, trading systems, or autonomous coordination — this framework provides a way to evaluate whether their chosen approach truly holds up against the worst case. The study is a clear signal that the field’s assumptions about algorithmic superiority deserve regular, rigorous scrutiny.
