Nova: Imagine you're watching a panel of experts on TV — economists, political pundits, foreign policy gurus — all confidently predicting what's going to happen next. Now imagine a dart-throwing chimpanzee. Which one do you think makes more accurate predictions?

Nova: Nova: That's not the setup to a joke. That's the punchline from one of the most devastating findings in the history of social science. Philip Tetlock, a psychologist at the University of Pennsylvania, spent twenty years tracking 284 experts making nearly twenty-eight thousand predictions — and the average expert was about as accurate as random chance. A dart-throwing chimp.

Nova: Nova: I'm Nova, and this is Aibrary. Today we're diving into Superforecasting: The Art and Science of Prediction by Philip Tetlock and Dan Gardner. A book that doesn't just demolish our illusions about expertise — it also reveals something genuinely hopeful. Some people really can see the future better than the rest of us. And their secret isn't genius. It's a set of habits anyone can learn.

Nova: It is absolutely real. Tetlock started his Expert Political Judgment study back in 1984, recruiting 284 people whose entire profession was commenting on or offering predictions about political and economic trends. Over two decades, they made 27,451 forecasts. And when Tetlock checked the results — the average expert barely did better than if you'd just flipped a coin.

Nova: Here's where it gets even more interesting. When Tetlock dug deeper, he found the experts actually split into two statistically distinguishable groups. One group did worse than the chimp. But another group consistently beat it — not by a wide margin, but repeatedly. So the question became: what was different about the second group?

Nova: Exactly. Tetlock discovered that forecasting isn't an inborn talent like perfect pitch. It's a skill, and it can be taught, practiced, and improved. His subsequent research — the Good Judgment Project — even found that ordinary volunteers, without security clearances, could outperform CIA intelligence analysts by thirty percent. All by applying a specific set of thinking techniques. Today we're going to unpack those techniques, and what they mean for how all of us make decisions.

Nova: None of those things. Tetlock found that what mattered wasn't what they thought — whether they were liberal or conservative, optimists or pessimists — or even whether they had PhDs. The critical factor was how they thought. And he used a famous metaphor to explain it, drawn from the essayist Isaiah Berlin.

Nova: You know it. Berlin borrowed a line from the ancient Greek poet Archilochus: the fox knows many things, but the hedgehog knows one big thing. Hedgehogs are the Big Idea people. They have one grand theory about how the world works — Marxism, libertarianism, whatever — and they squeeze every problem into that framework. They think in terms of certainties and impossibilities. They're deeply confident. And they make great television.

Nova: Right. And here's the brutal irony Tetlock discovered: when hedgehogs made forecasts on the subjects they knew the most about — their own specialties — their accuracy actually declined. The more famous the expert, the less accurate they were.

Nova: You'd think so. But the hedgehog's Big Idea acts like a pair of glasses they never take off. It distorts everything they see. They become more confident in their framework, more dismissive of contradictory evidence. Foxes, on the other hand, are scrappy and eclectic. They draw from many sources, many theories. They think in probabilities, not certainties. They admit mistakes and change their minds. And in Tetlock's data, foxes beat hedgehogs on both calibration and resolution — meaning they were better at assigning accurate probabilities, and better at distinguishing signal from noise.

Nova: Great question. That was the initial concern — maybe foxes just hugged the fifty percent line and looked reasonable by default. But no. The data showed foxes had genuine foresight. They could actually discriminate between things that were likely and unlikely to happen. The hedgehogs, with all their confidence, couldn't distinguish a sixty-forty bet from a forty-sixty one.

Nova: Tetlock has this great line: declarations of high confidence mainly tell you that an individual has constructed a coherent story in their mind — not that the story is true. The hedgehog is a master of internal coherence. The fox cares about external correspondence with reality.

Nova: This brings us to the research project that made superforecasting famous. In 2011, an organization called IARPA — that's the Intelligence Advanced Research Projects Activity, part of the Office of the Director of National Intelligence — launched a massive forecasting tournament. They wanted to know: can we systematically improve the accuracy of geopolitical forecasts? Four years, five hundred questions, over a million forecasts.

Nova: Exactly. Tetlock and his colleague Barbara Mellers at Penn created the Good Judgment Project. They recruited thousands of regular people online — not intelligence veterans, not policy wonks, just curious volunteers. These people made forecasts about things like: Will Greece leave the Eurozone? Will there be a significant military conflict in the South China Sea? Will North Korea test a nuclear weapon?

Nova: Not just beat — they crushed it. The Good Judgment Project won the tournament so decisively that IARPA effectively shut down the competition early. The top forecasters — the top two percent of the roughly one hundred thousand participants over time — were about thirty percent more accurate than intelligence analysts with security clearances. And when you aggregated their forecasts into teams, the advantage grew even larger.

Nova: It seems like it should be, right? But classified information often comes with its own biases. It's fragmentary, it's selected by someone else's judgment, and it can create an illusion of completeness. The superforecasters, by contrast, were combing through open-source information — news reports, economic data, academic research — but they were doing it with a set of cognitive habits that systematically reduced error.

Nova: Tetlock distilled them into what he calls the Ten Commandments for Aspiring Superforecasters. Let's walk through the most crucial ones. First: triage. Don't waste time on questions that are basically clocklike — where simple rules get you close — or cloudlike — where nothing works. Focus on the Goldilocks zone where effort pays off.

Nova: Exactly. But asking who might win this season and with what confidence? That's tractable. Second: Fermi estimation. Enrico Fermi, the physicist who built the first nuclear reactor, loved breaking down impossible-sounding questions — like how many piano tuners are in Chicago — into smaller, knowable components.

Nova: That's it. It flushes ignorance into the open. Better to expose your assumptions and be wrong quickly than to hide behind vague language. Third: balance the inside view and the outside view. Superforecasters always start with base rates. Before they ask what makes this situation unique, they ask: how often do things like this happen in situations like this?

Nova: Right. It's the antidote to what Daniel Kahneman calls the planning fallacy. You think your project is special. It probably isn't. Start with how similar projects have gone historically — then adjust for what's genuinely different. And fourth — and this one is absolutely critical: update your beliefs constantly, in small increments.

Nova: Precisely. Superforecasters treat beliefs as hypotheses to be tested, not treasures to be guarded. They update their probability estimates frequently — sometimes multiple times a week — and always in proportion to the weight of the new evidence. They don't overreact. They don't underreact. They nudge.

Nova: One of my favorite concepts is what Tetlock calls dragonfly eye. A dragonfly has compound eyes with thousands of lenses, each capturing a slightly different perspective, all synthesized into a single coherent image. That's what superforecasters do. They gather multiple perspectives — not just their own, not just the ones they agree with — and synthesize them.

Nova: Yes. The commandment says: look for the clashing causal forces at work in every problem. For every good argument, there is typically a counterargument worth acknowledging. Superforecasters don't just tolerate opposing views — they hunt for them. They can articulate the other side's argument so well that the other side would say, yes, that's exactly what I believe.

Nova: And that's why most of us are bad forecasters. But here's another crucial habit: superforecasters think in precise probabilities. The psychologist Amos Tversky once observed that most people have exactly three settings for probability: gonna happen, not gonna happen, and maybe. Superforecasters distinguish between fifty-five percent and sixty-five percent. They understand that those ten percentage points represent real differences in expected value.

Nova: It feels arbitrary because we're not used to it. But Tetlock's data shows that the more granular the probability estimate, the better the forecast. And superforecasters rarely use fifty percent. Fifty percent is what you say when you have no idea. Good forecasters have information, and their probabilities reflect that.

Nova: Completely. Because uncertainty isn't a sign of weakness — it's an ineradicable element of reality. Tetlock distinguishes between epistemic uncertainty — things you don't know but could in theory find out — and aleatory uncertainty, which is genuine randomness in the universe. Good forecasters know which is which. And they know that saying a thing is impossible or certain is almost never justified.

Nova: It is. And that's actually one of the key findings. Superforecasters score high on something called need for cognition — they actually enjoy thinking hard. Not everyone does. But Tetlock also found something even more predictive than intelligence.

Nova: The strongest predictor of rising into the ranks of superforecasters is something Tetlock calls perpetual beta — the degree to which one is committed to belief updating and self-improvement. It is roughly three times as powerful a predictor as its closest rival, which is intelligence.

Nova: Exactly. Superforecasters do score above the eightieth percentile on intelligence and numeracy tests, so you need a certain baseline. But the big jump in accuracy comes from the top performers, not from the already-smart group as a whole. And what sets those top performers apart is perpetual beta — treating yourself like software that's never in a final release, always being tested, analyzed, patched, and improved.

Nova: Precisely. Tetlock draws on Carol Dweck's work on growth mindset here. Superforecasters believe their abilities can be developed through effort. They don't think forecasting talent is fixed. And they pair that growth mindset with grit — the perseverance to keep going even after being wrong, which happens a lot.

Nova: Yes. This is absolutely crucial. In most punditry, nobody keeps score. The expert makes a confident prediction, it turns out wrong, and nobody mentions it — the conversation just moves on. Tetlock insists on using Brier scores, which measure the distance between what you forecast and what actually happened. Lower Brier score means better accuracy. And you need benchmarks: can you beat a dumb rule, like always saying fifty percent? Can you beat other forecasters?

Nova: Right. They conduct after-action reviews — not just on failures, but on successes too, because sometimes you were right for the wrong reasons. They keep decision journals. They want to know: where exactly did my reasoning break down? Was it a bad assumption? Did I miss a key piece of evidence? Did I overweight my own expertise?

Nova: Yes. Superforecasters update their probability estimates far more frequently than regular forecasters — sometimes multiple times a week — but always in small, incremental adjustments. They're not swinging wildly from ten percent to ninety percent on a single news story. They nudge their estimate from sixty to sixty-five based on a subtle signal. It's about teasing meaningful information from noisy news flows without overreacting.

Nova: That's exactly right. And Tetlock's final commandment captures this perfectly: master the error-balancing bicycle. You can't learn to ride by reading a physics textbook. You have to get on the bike, crash a few times, and feel the balance for yourself. Forecasting is the same — it requires deliberate practice with clear feedback.

Nova: Absolutely. And the team findings are fascinating. In the Good Judgment Project, teams were about twenty-three percent more accurate than individuals working alone. And superteams — the best-performing groups — beat prediction markets by fifteen to thirty percent.

Nova: Right. The wisdom of crowds works when you aggregate independent judgments. But teams interact — and interaction can go very wrong. Think of groupthink, or the Bay of Pigs fiasco. The same team that bungled the Bay of Pigs performed brilliantly during the Cuban Missile Crisis. The difference wasn't the people. It was the process.

Nova: Tetlock identifies three key skills. First, perspective taking — understanding the arguments of the other side so well that you can reproduce them to the other side's satisfaction. Second, precision questioning — helping others clarify their arguments so they're not misunderstood. And third, constructive confrontation — learning to disagree without being disagreeable.

Nova: They do. And superforecasters have an advantage here: because they're often not professional experts in the domain they're forecasting, they have less ego invested. They don't have a career's worth of reputation riding on being right about a particular theory. They can treat their beliefs as hypotheses.

Nova: Yes — Tetlock calls this the leader's dilemma. How can a leader inspire confidence if they see nothing as certain? The answer draws on a military doctrine called Auftragstaktik, or mission command. The idea is that commanders tell subordinates what the goal is, but not how to achieve it. The fundamental message is: think. If necessary, discuss or criticize orders. If you absolutely must, disobey them. But once a decision is made — once the moment comes to act — then you must set aside uncertainty and complexity and commit fully.

Nova: Exactly. Intellectual humility isn't the same as self-doubt. It's possible to think highly of yourself and remain intellectually humble. The best leaders know that no plan survives contact with reality. They plan for surprise. They build flexibility into their systems. And they surround themselves with fox-like thinkers who won't just tell them what they want to hear.

Nova: Tetlock has this wonderful observation: the more famous an expert, the less accurate they tend to be. Fame selects for hedgehogs — for people who tell clear, confident stories. But accuracy selects for foxes — for people who are messy, provisional, and self-critical. The book is essentially an argument that we need to rewire our intuitions about who deserves our trust.

Nova: That's a great summary. And I'd add: it's also about knowing which questions to tackle and which to leave alone — the Goldilocks zone where effort actually pays off. Plus keeping rigorous score. Without feedback, you can't improve. Most experts never get meaningful feedback on their predictions, which is why they can be wrong for decades without ever noticing.

Nova: Tetlock's research suggests that while not everyone will reach the top two percent, almost anyone can improve. Even a sixty-minute tutorial on basic forecasting principles improved accuracy by about ten percent in the Good Judgment Project. That's not nothing. And the book is full of stories of ordinary people — a retired computer programmer, a homemaker, a pharmacist — who became world-class forecasters simply by practicing these techniques deliberately over time.

Nova: Exactly. Whether you're deciding on a career move, evaluating a business strategy, or just trying to figure out what's going to happen in the world, the superforecasting mindset helps. Be less certain than you want to be. Look for what you might be missing. Put a number on your uncertainty, even if it feels uncomfortable. And when you're wrong — which you will be, often — treat it as data, not failure.

Nova: And here's a final thought that I find genuinely moving. Tetlock's research began as a kind of debunking project — showing that experts aren't nearly as good as they claim to be. But it evolved into something deeply optimistic. The discovery that ordinary people, using nothing but open-source information and the right thinking habits, could outperform the intelligence community — that's evidence that foresight is real. It's limited, it's probabilistic, it works better in the short term than the long term. But it's real. And we can all get better at it.

Nova: This is Aibrary. Congratulations on your growth!