Dr. Roland Steele: In 2007, Microsoft's CEO Steve Ballmer famously laughed at the iPhone, declaring, 'There's no chance it's going to get any significant market share.' He was a brilliant, successful leader, yet his prediction is now a legendary blunder. This wasn't a unique slip-up; it's a pattern. So-called experts, from CEOs to political pundits, are often spectacularly wrong. But what if there's a small group of ordinary people—a retired government employee, a pharmacist, a housewife—who can consistently predict the future better than these experts and even intelligence analysts with access to classified data? What do they know that the rest of us don't?

fish: That’s the million-dollar question, isn't it? It makes you wonder if there's a fundamental flaw in how we approach 'knowing' things.

Dr. Roland Steele: It absolutely is. And in their book 'Superforecasting,' Philip Tetlock and Dan Gardner pull back the curtain on this exact question. Today we'll dive deep into this from two perspectives. First, we'll expose the 'accountability crisis' in forecasting and find out why so many experts get away with being wrong. Then, we'll open up the superforecaster's toolkit and learn the practical, systematic methods they use to achieve stunning accuracy. And joining me is fish, a curious and analytical thinker whose background in design and customer support gives them a unique appreciation for systems and clear outcomes. fish, it's great to have you.

fish: Thanks, Roland. That iPhone example is perfect. It makes you wonder if there's a fundamental flaw in how we approach 'knowing' things. I'm fascinated by the idea of a more rigorous, almost inspectable, way of thinking.

Dr. Roland Steele: Exactly! 'Inspectable' is the perfect word. And that brings us to our first big idea: the accountability crisis. The book argues that most 'expert' forecasting isn't designed to be right; it's designed to sound good, to be confident, to support a narrative.

fish: It’s performance, not prediction.

Dr. Roland Steele: That's it exactly. And the book uses a brilliant analogy from the philosopher Isaiah Berlin to explain this. He divided thinkers into two categories: Hedgehogs and Foxes. Hedgehogs know one big thing. They have a grand theory, an ideology, that they apply to everything. Think of a die-hard free-marketer or a staunch socialist. Everything is viewed through that single lens.

fish: They have their one hammer, and every problem looks like a nail.

Dr. Roland Steele: Perfectly put. Foxes, on the other hand, know many little things. They're skeptical of grand theories. They gather information from all sorts of different sources, they're self-critical, and they're comfortable with nuance and complexity. And Tetlock's decades of research show, unequivocally, that Foxes are far, far better forecasters than Hedgehogs.

fish: So the Hedgehogs are the confident pundits we see on TV, and the Foxes are the quiet, methodical people who are actually getting it right.

Dr. Roland Steele: You've got it. And a classic case of a Hedgehog in action is the economist Larry Kudlow. Kudlow was a huge proponent of supply-side economics—the 'one big idea' that tax cuts always lead to economic growth. When President George W. Bush enacted big tax cuts in the early 2000s, Kudlow confidently and repeatedly predicted a massive 'Bush Boom.'

fish: And I'm guessing that boom didn't quite materialize as planned?

Dr. Roland Steele: Not even close. For years, even as the economy showed disappointing growth and was clearly heading towards the 2008 financial crisis, Kudlow insisted the boom was happening. He called it 'the biggest story never told.' In December 2007, the very month the Great Recession officially began, he wrote, 'There is no recession... we are about to enter the seventh consecutive year of the Bush boom.' He was a classic Hedgehog—so attached to his one big idea that he was incapable of processing contradictory evidence.

fish: That's fascinating. So his failure wasn't just a bad prediction, it was a failure of process. He wasn't updating his beliefs. In customer support, if a script isn't working and satisfaction scores are dropping, you don't insist the script is perfect; you analyze the data and change it. Kudlow was ignoring his 'customer satisfaction' data.

Dr. Roland Steele: Precisely! He had no incentive to. His job was to be a confident Hedgehog, to tell a good story. And the book's solution is to be more like a Fox—and to keep score. Superforecasters attach specific probabilities to their predictions, like 'there's a 70% chance this will happen,' and then they use something called a Brier score to track their accuracy over time. It forces honesty.

fish: So it's about creating a system of accountability for yourself. It removes the ego and the storytelling and just focuses on the cold, hard facts of your track record. That's a very logical, almost engineering-like approach to judgment.

Dr. Roland Steele: It is. It’s about making your own thinking transparent and testable. You can’t improve what you don’t measure. Hedgehogs never measure. Foxes live by their measurements.

fish: It makes sense. In design, you have user testing. You don't just assume your design is intuitive. You watch people use it, you collect data, and you find the flaws. You have to be willing to be proven wrong to make the product better. It sounds like the same principle.

Dr. Roland Steele: That's a perfect parallel. It's about a fundamental commitment to reality over narrative.

Dr. Roland Steele: And that engineering approach you mentioned, fish, is the perfect transition to our second key idea: the actual tools in the superforecaster's toolkit. This isn't about having a crystal ball; it's about having a better method. And one of the most powerful is a technique called 'Fermi-izing'.

fish: Fermi-izing? Like the physicist, Enrico Fermi?

Dr. Roland Steele: The very same. He was a master of what are now called Fermi estimations. He once asked his students a seemingly impossible question: 'How many piano tuners are there in Chicago?' They had no data, no internet, nothing. It seems like a pure guess.

fish: Right, where would you even begin?

Dr. Roland Steele: Well, Fermi showed them. You don't guess the final number. You break the question down into a series of smaller, more manageable, and estimable parts. First, what's the population of Chicago? Let's say it's 3 million. How many people per household? Maybe two or three. So that's about a million households. What percentage of households would own a piano? It's not a common item, so maybe one in twenty? That's 50,000 pianos.

fish: Okay, I see where this is going. You're building a logical chain.

Dr. Roland Steele: Exactly. Then, how often are pianos tuned? Maybe once a year. How long does a tuning take? Let's say two hours. And how many hours does a full-time piano tuner work in a year? Maybe 2,000 hours. So, you have 50,000 pianos needing two hours of work each, which is 100,000 hours of tuning work per year. Divide that by the 2,000 hours one tuner works, and you get... 50 piano tuners. Suddenly, this huge unknown becomes a series of small, reasonable guesses. And the amazing thing is, their final estimate was shockingly close to the actual number listed in the phone book at the time.

fish: I love that. It's a structured way to deal with ambiguity. It turns a big, scary 'I don't know' into a series of manageable 'what's a reasonable assumption here?'. In design, you might use this to estimate the time for a complex project you've never done before. You don't just guess 'six weeks.' You break it down: How long for wireframes? For mockups? For user testing? For revisions? It makes the unknown knowable.

Dr. Roland Steele: It's a beautiful, logical process. And this ties directly into the superforecaster mindset, which the book calls being in 'perpetual beta.' It’s a term from the software world, meaning a product is never truly finished; it's always being updated and improved.

fish: Always a work in progress.

Dr. Roland Steele: Exactly. Superforecasters see their own judgment the same way. They see a failure not as a personal flaw, but as a bug in their forecasting software that needs to be patched. The book tells the story of Mary Simpson, an economist who was devastated that she completely missed the warning signs of the 2008 financial crisis. But instead of giving up, she used that failure as motivation. She joined Tetlock's project and became one of the top superforecasters. She was determined to patch her own mental software.

fish: So, they're always in a state of improvement. They're never 'done' learning. That resonates deeply. You're never a 'perfect' designer or a 'perfect' customer support agent. You're always learning from the last interaction or the last project. The goal isn't to be right all the time, but to get less wrong over time.

Dr. Roland Steele: That's the entire philosophy in a nutshell. 'Getting less wrong over time.' It's about embracing the process of improvement, which requires humility and a lot of hard work.

Dr. Roland Steele: So, to bring it all together, we've seen that the world of expertise is often a world of unaccountable Hedgehogs, who prioritize a good story over the truth. But we can choose to be Foxes.

fish: And being a Fox isn't about being a genius. It's about having a better process: making your judgments testable, breaking down problems with tools like Fermi-izing, and having the humility to be in 'perpetual beta,' always learning from the data.

Dr. Roland Steele: Absolutely. It's a skill, not a gift. And it's a skill anyone can start to build. So, for everyone listening, here's a simple challenge from the book. The next time you feel confident about something—who will win a game, whether a project will finish on time, anything—don't just think it. Take 10 seconds to put a number on it. 'I'm 70% sure this will happen.' Write it down.

fish: And then, crucially, check the result later. Don't just forget about it. That simple act of writing it down and checking it is the first step. It's how you start building your own, personal system for better judgment. It’s how you become your own inspector.

Dr. Roland Steele: Become your own inspector. I love that. It’s the beginning of a fascinating journey into how you think. fish, thank you so much for bringing your sharp, systematic perspective to this.

fish: It was a pleasure, Roland. It's given me a lot to think about.