Based on the latest Radical Talks episode with Rich Kotite and Aaron Rosenberg, Partners at Radical Ventures, and Mantic co-founders Toby Shevlane and Ben Day. Listen on Spotify, Apple Podcasts, or YouTube.

Last month, in “The Era of Superhuman Forecasting” Rich Kotite and Aaron Rosenberg shared why Radical Ventures led Mantic’s seed round. In our latest Radical Talks episode, Rich and Aaron dove deeper, speaking with co-founders Toby Shevlane and Ben Day about how the system keeps improving, why forecasting has unusual economics, and what it takes to turn a probability into a decision.

Getting the Forecast Right

The story starts at Google DeepMind. Toby was working on what the organization’s five-year plan should be as AI advanced and intersected with geopolitics. It was the kind of problem where adding one more analyst wouldn’t do much. “I wanted it to be like all the world’s experts thinking for 100 years about what’s going to happen in the next five years and what should we do,” he said. Building that system meant solving three problems: making forecasts accurate, making them economical, and making them useful.

In early 2024, a UC Berkeley paper showed promising results from testing language models on past forecasting questions. The paper’s method, backtesting, became Mantic’s foundation. Because the system is scored on questions whose outcomes are already known, the team can quickly measure the effect of every change, from a new model to a new data source. Comparing methods with human forecasters can take a year. “We can do that in a morning,” Ben said.

Rapid testing also showed what makes a strong forecasting system. Combining frontier models works best when the models disagree. In one experiment, a less accurate Grok model added more to the group than stronger models did, because its forecasts differed from theirs. Mantic fine-tunes its own model to cover the blind spots of the others. Ben credits this steady compounding for Mantic’s result in this summer’s Metaculus Cup — outscoring all 676 human participants, including professional forecasters.

Making It Economical

Accuracy raised a second question: how much should a forecast cost? Ben compares forecasting to the weather. Millions of people want the same weather forecast because they live in the same place, which is why it makes sense to run supercomputers around the clock. Many of the questions Mantic’s customers care about work the same way. Mantic can spend far more computing power on each forecast than any single user would, and still sell it for less than it would cost that user to produce it themselves.

That leaves room for spending to grow on the questions that matter most. For example, Toby shared, “What would it look like to spend ten million dollars of inference on ‘When is the Strait of Hormuz going to reopen?’” 

Turning Forecasts Into Decisions

The final challenge is helping customers act on a forecast. A probability alone can’t answer that. Toby describes Mantic’s product as an onion. The probability sits at the core, surrounded by the range of possible outcomes, how the forecast has moved over time, and the reasoning behind it. “Really what we’re giving to people is not just a percentage, it’s like a model of the world,” he said. Users can also ask conditional questions, and Mantic is adding a way for customers to layer in their own insights. This shows them whether their private knowledge changes the forecast or is already priced in.

More and more, those forecasts reach customers inside the tools they already use. After Mantic launched its MCP server, one private equity partner’s usage rose sharply, and he now uses his own AI assistant to flag risks for the companies in his portfolio. Hedge funds remain Mantic’s core market because they make frequent, high-stakes decisions and are comfortable acting on quantitative information. Toby sees the next step as expanding to whoever looks most like a trader, such as corporate finance teams.

Every major decision rests on a forecast. Mantic is making those forecasts measurable, affordable, and actionable.

Radical Talks is a podcast from Radical Ventures exploring innovation at the forefront of AI. Subscribe on Apple Podcasts, Spotify, or YouTube.