Mark: A 20-year study of 284 experts—people who advise presidents and run global economies—found their predictions were only slightly more accurate than a chimpanzee throwing darts at a dartboard. Today, we find out who actually can see the future. Michelle: Hold on, a dart-throwing chimp? That’s both insulting and hilarious. So you’re telling me my uncle who yells at the TV during election season might have a point after all? Mark: He might be closer than you think! That shocking finding comes from the life's work of Philip Tetlock, who, along with journalist Dan Gardner, wrote the book we're diving into today: Superforecasting: The Art and Science of Prediction. Michelle: And this isn't just some academic thought experiment. Tetlock's research got so much attention that the US intelligence community actually funded his follow-up project to see if a group of ordinary volunteers could beat their own professional analysts. Mark: Exactly. It was a massive, multi-year tournament born from the ashes of major intelligence failures, like the WMD fiasco before the Iraq War. The book is widely acclaimed, though some critics, like Nassim Taleb of 'Black Swan' fame, have raised some very pointed questions we'll get into. But it all starts with one fundamental problem...

Mark: So what's going on here? Why are the people we pay to be right—the high-profile pundits, the talking heads on TV, the economists with fancy degrees—so often wrong? Michelle: I’ve always assumed their job is more about being confident and entertaining than being correct. No one wants to watch a pundit say, "Well, there's a 43% chance of a recession, but I could be wrong." It's not great television. Mark: That's a huge part of it. Tetlock has a brilliant metaphor for this, borrowed from the philosopher Isaiah Berlin. He says thinkers are either "hedgehogs" or "foxes." Michelle: Okay, I'm intrigued. Hedgehogs and foxes? What does that mean? Mark: A hedgehog knows one big thing. They have a grand theory, a single lens through which they see the world. Think of a die-hard free-market capitalist who believes deregulation is the answer to everything, or a staunch Marxist who sees class struggle behind every event. They are ideologues. Michelle: Right, they have their one big hammer, and every problem looks like a nail. Mark: Precisely. And they're confident, they're decisive, they tell a clean, simple story. The media loves them. The problem is, the world is messy and complex, so their one big idea is often wrong. A fantastic example from the book is the economist Larry Kudlow. Michelle: Oh, I know that name. He’s been a prominent voice for decades. Mark: A classic hedgehog. He’s a champion of supply-side economics—the idea that cutting taxes will unleash massive economic growth. When President George W. Bush enacted big tax cuts in the early 2000s, Kudlow was certain a historic boom would follow. He called it the "Bush Boom." Michelle: And how did that turn out? Mark: Well, the economy grew, but it was pretty underwhelming compared to past booms. But Kudlow was undeterred. Year after year, he insisted the boom was happening, calling it "the biggest story never told." In December 2007, with the first tremors of the financial crisis already being felt, he wrote, "There is no recession. In fact, we are about to enter the seventh consecutive year of the Bush boom." Michelle: Wow. That is… a spectacular miss. The Great Recession officially began that exact month. Mark: A perfect hedgehog moment. The facts changed, but his Big Idea didn't. He couldn't see the data that contradicted his worldview. Now, contrast that with the "fox." Michelle: Let me guess, the fox is more cunning? Mark: More… eclectic. The fox knows many little things. They're not committed to one grand theory. They take ideas from different disciplines, they're comfortable with nuance and contradiction, they gather evidence from many sources, and most importantly, they change their minds when the facts change. Michelle: That sounds a lot less certain. And probably less likely to get a prime-time cable news show. Mark: Exactly! They're more likely to say, "Well, on the one hand, this could happen, but on the other hand..." They use probabilities. They're self-critical. And Tetlock's data showed, overwhelmingly, that foxes are far, far better forecasters than hedgehogs. Michelle: Okay, the hedgehog idea is catchy. But isn't that just stubbornness? What's the deeper psychology here? Why are our brains so drawn to that flawed hedgehog thinking? Mark: It's about how our brains are wired. The book leans heavily on the work of Daniel Kahneman, another giant in this field. He talks about two modes of thinking: System 1 and System 2. Michelle: I think I’ve heard of this. System 1 is the fast, intuitive, gut-reaction part of our brain, and System 2 is the slow, deliberate, analytical part? Mark: You got it. System 1 is what helps you drive a car without thinking about every single action. It's automatic. System 2 is what you use to solve a difficult math problem. It takes effort. The problem is, our brains are lazy. System 2 is a huge energy hog, so we default to System 1 whenever we can. Michelle: The brain's on a permanent energy-saving mode. I can relate. Mark: And that's where the trouble starts. Let me give you a quick puzzle from the book. A bat and a ball together cost $1.10. The bat costs one dollar more than the ball. How much does the ball cost? Michelle: Uh… ten cents. Mark: That's what almost everyone says instantly. It's the System 1 answer. It feels right. But it's wrong. Michelle: Wait, what? How? Mark: If the ball is ten cents, and the bat is a dollar more, the bat would be $1.10. Together, they'd be $1.20. The correct answer is that the ball costs five cents, and the bat costs $1.05. Michelle: Oh, man. You got me. My lazy brain just signed off on the easy answer without doing the work. Mark: That's the core of the problem! Hedgehogs live in System 1. They jump to conclusions based on their Big Idea, and their System 2 is too lazy or too biased to check the work. They suffer from confirmation bias—they only look for evidence that supports their theory. They have what Tetlock calls an "illusion of knowledge." Michelle: So the experts are overconfident hedgehogs running on lazy brain software. That's a bit depressing. Who, then, are these people who actually get it right? Are they all super-geniuses with PhDs in quantum physics? Mark: That’s the most hopeful and surprising part of the book. The answer is a resounding no.

Mark: The people who beat the experts, who beat the dart-throwing chimps, who even beat intelligence analysts with access to classified information, are what Tetlock calls "superforecasters." And they are, for the most part, ordinary people. Michelle: Ordinary how? Mark: Take Bill Flack, one of the stars of the book. He’s a retired 55-year-old employee of the US Department of Agriculture, living in Kearney, Nebraska. He spends his free time forecasting global events on his computer. Michelle: So not exactly a Davos Man jet-setting with world leaders. Mark: Not at all. And that's the point. His power doesn't come from what he knows or who he knows. It comes from how he thinks. He's a quintessential fox. He's curious, open-minded, and comfortable with numbers and probabilities. Michelle: Okay, so what does that look like in practice? Give me an example of how a superforecaster tackles a really hard question. Mark: A great one from the book is the question of whether Yasser Arafat, the Palestinian leader, had been poisoned with polonium. This came up years after his death when traces were found on his belongings. The question in the tournament was: would official inquiries find elevated levels of polonium in his exhumed remains? Michelle: That is a tough one. My gut reaction would be to just guess based on my feelings about the Israeli-Palestinian conflict. It feels like a political question. Mark: And that's the trap! That's the hedgehog approach. A hedgehog would say, "Of course Israel poisoned him!" or "That's a ridiculous conspiracy theory!" and anchor their forecast to that belief. But Bill Flack, the superforecaster, did something completely different. He started breaking the problem down. Michelle: He "Fermi-ized" it, as the book says. Named after the physicist Enrico Fermi. Mark: Exactly. He asked a series of smaller, more manageable questions. First, the science: What is polonium-210? What's its half-life? Could it still be detectable after all these years? Then, the logistics: Who is doing the testing? Are they credible? Could the samples be contaminated? He even looked into the possibility that Arafat was a heavy smoker and that polonium is found in tobacco, and wondered if that could be a confounding factor. Michelle: Wow. He completely stripped the politics out of it and turned it into a science problem. Mark: He turned a "who is to blame" question into a "what is the evidence" question. He started with an initial probability, and as new information came in—like the Swiss testing lab announcing a delay—he would make small, careful updates to his forecast. He wasn't making wild swings based on headlines; he was making tiny, incremental adjustments based on the weight of new evidence. Michelle: This is fascinating. It's not about having a crystal ball. It's a skill. A process. It's about being a better detective. Mark: A better probabilistic detective. That's the key. They don't think in terms of "yes" or "no." They think in terms of 60%, 70%, 85%. They embrace uncertainty and try to quantify it. Michelle: So this 'Fermi-izing'—is it just breaking a big scary question into a bunch of smaller, less-scary questions? Mark: That's a perfect way to put it. It's a way to combat the "illusion of knowledge." Instead of pretending you know the answer to a huge, complex question, you admit you don't. But you identify the smaller pieces you can make educated guesses about. Michelle: It’s like instead of asking 'Will I be happy in 5 years?', which is impossible to answer, you ask, 'What are the 10 small things that contribute to my happiness, and what's the probability I can improve each one by 10%?' Mark: You're a natural superforecaster, Michelle! And that process of constant questioning and adjusting is the final piece of the puzzle. It's a whole mindset that Tetlock calls being in 'perpetual beta.'

Michelle: 'Perpetual beta.' I like that. It sounds like something from Silicon Valley. Like a product that's never finished, always being improved. Mark: That's exactly the idea. Superforecasters see their own beliefs as being in perpetual beta. They are never "finished" or "final." They are hypotheses to be tested, not treasures to be guarded. This requires two crucial ingredients: grit and a growth mindset. Michelle: A growth mindset being the belief that your abilities aren't fixed, that you can get smarter and better through effort. Mark: Right. And the book has a powerful story about this. Mary Simpson, an economist with a PhD, was working as a financial consultant. And she completely missed the signs of the 2008 financial crisis. It was a huge blow, not just to her finances, but to her professional identity. Michelle: I can imagine. That would be like a meteorologist getting caught in a hurricane they didn't see coming. Mark: She said, "I really totally missed it... and that was frustrating to me because I have the background to understand what went wrong." But instead of getting defensive or giving up, she got gritty. She said, "I should be better at this," and she joined the Good Judgment Project with the explicit goal of learning from her failure and becoming a better forecaster. Michelle: And did she? Mark: She became one of the top superforecasters in the entire tournament. Her failure became the fuel for her growth. That's the perpetual beta mindset. It's about analyzing your mistakes without flinching, and analyzing your successes too, to see if you just got lucky. Michelle: That brings up a really important point. How much of this is just luck? Maybe these superforecasters just had a good run. Mark: Tetlock is obsessed with that question. He ran the numbers, accounting for regression to the mean—the statistical tendency for unusually good or bad performance to be followed by more average performance. And he found that while luck is a factor, it doesn't explain the superforecasters' persistent success. Their skill is real and measurable. Michelle: Okay, I'm sold on the method for individuals. But what about groups? We all know how meetings can turn into groupthink disasters. Does putting a bunch of superforecasters in a room together make them even smarter, or does it just create a super-disaster? Mark: That's one of the most interesting findings. When they put superforecasters on teams, their accuracy shot up even more. They outperformed prediction markets, which are often seen as the gold standard of aggregating information. Michelle: How? What made their teams work so well? Mark: It was the culture. They created what the book calls "constructive confrontation." They would challenge each other's reasoning, but always respectfully. They'd share information, point out flaws in each other's logic, and synthesize their different views. It's the opposite of groupthink. The book contrasts the disastrous groupthink of the Bay of Pigs invasion with the successful, debate-driven process Kennedy's team used during the Cuban Missile Crisis just a year later. Same people, different process, dramatically different outcome. Michelle: That's a powerful example. But I have to ask the big, skeptical question. The one that I know some of our listeners are thinking. Mark: I think I know where you're going. The Black Swan. Michelle: Exactly. This is the main critique from people like Nassim Taleb. This is all well and good for forecasting elections or economic trends. But can superforecasting predict a 9/11? A global pandemic? A stock market crash? The truly massive, unpredictable events that really change the world? Or is this whole exercise just rearranging deck chairs on the Titanic? Mark: It's the most important critique, and Tetlock addresses it head-on. The answer is no, you cannot predict a specific Black Swan event. By definition, it's an outlier, something that's never happened before in that way. There's no data to model. Michelle: So Taleb is right? Forecasting is a fool's errand? Mark: Not so fast. Tetlock's response is nuanced and brilliant. He says you can't predict the swan, but you can forecast its consequences. You couldn't have predicted a novel coronavirus emerging in Wuhan in late 2019. But once it emerged, could you have forecasted the probability of it spreading globally? The likelihood of different government responses? The potential economic impact? Yes. Superforecasters are good at tracking the ripples after the stone is thrown in the pond. Michelle: I see. So it's not about predicting the lightning strike, but about predicting the probability of a forest fire once the lightning has struck. Mark: A perfect analogy. It’s about building resilience and adaptability. As Eisenhower said, "Plans are useless, but planning is indispensable." The act of forecasting, of thinking through possibilities, prepares you for a range of futures, even the ones you can't specifically predict.

Michelle: So when you strip it all away, this isn't really a book about having a magic 8-ball to predict the future. Mark: No, it's a book about thinking better. It's about replacing the arrogance of certainty with the power of curiosity. The real takeaway isn't a secret formula for seeing what's next; it's a mental toolkit for navigating the fog of uncertainty, whether you're a CEO, an investor, or just trying to figure out your own life. Michelle: It’s about intellectual humility. Acknowledging that the world is incredibly complex and that our own judgment is flawed, but that we can get better with deliberate practice. Mark: Exactly. It's about being a fox, not a hedgehog. It's about being comfortable saying "I think there's a 65% chance of this happening," and being even more comfortable changing that to 55% tomorrow when new evidence comes in. It's about being in perpetual beta. Michelle: I love that. So for everyone listening, what's one small 'superforecaster' habit they can start today to put this into practice? Mark: It's simple but surprisingly hard. Next time you have a strong opinion about something—politics, sports, a decision at work—just pause and ask yourself one question: "What would it take to make me change my mind?" Michelle: Ooh, that's a tough one. Mark: It is! But just trying to answer it forces you out of hedgehog mode. It makes you identify your own assumptions. Write down the answer. That's the first step to thinking like a superforecaster. It's the beginning of treating your beliefs as hypotheses to be tested, not facts to be defended. Michelle: That's a fantastic challenge. We'd love to hear what you all think. Find us on our socials and share one belief you're willing to put to the test. Let's get a conversation going and see if we can build a little superforecasting team of our own. Mark: This is Aibrary, signing off.