Prof. Eleanor Hart: As a software engineer, Yang, you spend your days building complex systems, writing code to solve problems. But what if I told you the most sophisticated learning algorithm ever deployed wasn't developed at Google or MIT, but in a tiny worm 600 million years ago?

hl01 yang: That's a bold claim, Eleanor. I'm intrigued. You're saying the best code is... biological?

Prof. Eleanor Hart: In a way, yes. It's the central idea in Max Bennett's fascinating book, "A Brief History of Intelligence." He argues that our intelligence isn't one single thing, but a series of five major evolutionary breakthroughs, like five major software updates, each building on the last. And understanding that ancient code, that evolutionary journey, might be the key to unlocking the future of artificial intelligence.

hl01 yang: I love that framing. Reverse-engineering the ultimate legacy system.

Prof. Eleanor Hart: Exactly. So today, we're going to dive into that idea and reverse-engineer our own minds. First, we'll explore the 'original source code' of intelligence—the simple, powerful act of steering. Then, we'll jump forward to a major 'upgrade'—how brains learned to learn from the consequences of their actions, a breakthrough that uncannily mirrors some of the most exciting developments in AI today.

hl01 yang: This sounds fantastic. It feels like it connects directly to the core challenges we face in tech—how to build systems that don't just calculate, but actually and.

Prof. Eleanor Hart: Precisely. And that's why I'm so glad you're here, Yang. Your perspective as an engineer is the perfect lens through which to view this.

Prof. Eleanor Hart: So let's start at the very beginning, before brains were even complex. The first problem life had to solve wasn't chess or language, but simply... which way to go. How do you move towards something good and away from something bad?

hl01 yang: The most basic decision-making. In programming, that's your fundamental if-else statement. If food, then approach. Else, avoid.

Prof. Eleanor Hart: You've nailed it. And the book gives this perfect, almost primal example: the nematode worm,. It has a tiny brain, just 302 neurons. Scientists put it in a petri dish with a bit of food on the other side. Now, you'd think it would just make a beeline for the food, but it doesn't.

hl01 yang: So it's not running a GPS calculation to find the most direct route?

Prof. Eleanor Hart: Not at all. It does something much simpler, and in a way, more elegant. It starts moving, and as it moves, it's constantly sensing. If the smell of the food gets stronger, it keeps turning in that general direction. If the smell gets weaker, it changes course and tries a new direction. It's not navigating to a point; it's just following a gradient. It's a simple, circling-in process.

hl01 yang: That's fascinating. From a coding perspective, that's a brilliant optimization algorithm. It's computationally cheap but incredibly effective. It doesn't need a complex world model or a map. It just needs one bit of information: is the signal getting better or worse? It's a simple loop: 'Is the signal stronger? If yes, continue. If no, change direction.'

Prof. Eleanor Hart: And Bennett calls this the first breakthrough: Steering. But the truly revolutionary part, the core of this first algorithm, is what he calls. The brain had to invent the concepts of 'good' and 'bad.' It had to assign a value to a stimulus. That food smell is 'good,' so approach. The smell of a predator is 'bad,' so avoid. This simple binary categorization is the 'Hello, World!' program of all decision-making.

hl01 yang: So valence is like the positive or negative weight assigned to a variable. And the steering algorithm just tries to maximize the positive and minimize the negative.

Prof. Eleanor Hart: Exactly. But here's where it gets another layer of sophistication, even in that simple worm. The book explains that this valence isn't fixed. For example, a hungry nematode is attracted to a whiff of carbon dioxide, because it could signal bacteria, which is food. But a well-fed nematode? It actively avoids CO2, because it could also signal the presence of a predator.

hl01 yang: So the system's internal state changes the value of the variable. The if statement isn't static. It's more like if else if . That's a much more dynamic and adaptive system. It's not just reacting, it's reacting based on its own needs.

Prof. Eleanor Hart: You see? The foundational code was already surprisingly complex. It wasn't just about the outside world; it was about the interplay between the world and the organism's internal state. And that ability to change what's considered 'good' or 'bad' is the perfect bridge to our next topic.

Prof. Eleanor Hart: Because what happens when the 'good' thing, the reward, doesn't come immediately after the action? If you perform a hundred small actions, and only at the very end you get a reward, how does your brain know which of those hundred actions was the crucial one?

hl01 yang: That's the credit assignment problem. It's a classic, and notoriously difficult, problem in machine learning. If a program makes a thousand moves to win a game of chess, you can't just reinforce the final 'checkmate' move. You have to figure out which moves, maybe hundreds of steps earlier, set up the win.

hl01 yang: So, if it got out of the maze, it would reinforce the actions it just took.

Prof. Eleanor Hart: That was the idea. But it was a spectacular failure. The SNARC would wander the maze, and if it eventually, by sheer luck, found the exit, it would try to reinforce all the recent synaptic firings. But it had no way of knowing which of its dozens of turns was the one. It was reinforcing noise. It couldn't solve the temporal credit assignment problem. For years, this was a major roadblock for AI.

hl01 yang: So how did AI, and for that matter, how did brains, solve this?

Prof. Eleanor Hart: The breakthrough, in both fields, was an idea called Temporal Difference Learning, or TD Learning, proposed by Richard Sutton in the 80s. It's a beautifully elegant solution. Instead of waiting for the final, actual reward, the system learns to the future reward at every step.

hl01 yang: Ah, so it creates its own intermediate rewards.

Prof. Eleanor Hart: Exactly! Sutton's model, often called an actor-critic model, has two parts. The 'critic' is constantly predicting the value of the current situation. 'From here, my chance of winning is 70%.' The 'actor' takes an action. Then the critic looks at the new situation. 'Okay, now my chance of winning is 80%.' That in predicted reward—that positive temporal difference—is the signal used to reinforce the actor's last move. You don't have to wait for the win; you just reinforce actions that make things.

hl01 yang: That is elegant. From a systems design perspective, it decouples the action from the final outcome and links it to a continuous stream of predictive feedback. It makes the learning process so much more efficient. It's no wonder this model is still so fundamental to reinforcement learning today.

Prof. Eleanor Hart: It is. An AI called TD-Gammon, using this exact principle, taught itself to play backgammon at a world-champion level in the 90s, just by playing against itself. But here is the most mind-blowing part of the book, the connection that ties it all together. For decades, neuroscientists thought dopamine was the brain's "pleasure chemical." But a series of experiments, particularly by Wolfram Schultz, showed something different.

hl01 yang: Let me guess. It's not about pleasure.

Prof. Eleanor Hart: It's not! Schultz found that in monkeys, dopamine neurons don't fire when the monkey gets an expected reward. They fire when the reward is. And they fire even earlier, at the cue that the reward. If the predicted reward doesn't show up, dopamine levels actually drop.

hl01 yang: Wait a second. That sounds exactly like the temporal difference signal. It's not signaling the reward; it's signaling the —the difference between what was expected and what actually happened.

Prof. Eleanor Hart: You got it. The brain's dopamine system a TD learning algorithm. It's the critic, broadcasting a signal that says 'this was better than expected, reinforce that behavior' or 'this was worse, don't do that again.'

hl01 yang: Wow. So biology and AI, separated by hundreds of millions of years, arrived at the same elegant solution to the same fundamental problem. That's... that's a case of convergent evolution, but for algorithms. It suggests there are fundamental, mathematical truths to how a complex learning system to work, whether it's built from silicon or from carbon.

Prof. Eleanor Hart: And that's the journey, isn't it? In just these first two breakthroughs, we've gone from a simple, reactive 'if-then' steering mechanism in a worm to a sophisticated, predictive learning system in vertebrates that's mirrored in our most advanced AI.

hl01 yang: It really is like an evolutionary software stack. You can't have the complex reinforcement learning module without the foundational 'valence' module that defines what's even worth reinforcing. Each layer is built upon the logic of the one before it.

Prof. Eleanor Hart: I think for anyone in tech, like yourself, Yang, the message from Bennett's book is incredibly powerful. To build the next generation of truly intelligent systems, maybe we don't just need bigger models and more data. Maybe we need a deeper appreciation for the architectural principles that evolution, the ultimate tinkerer, has already discovered and battle-tested over eons.

hl01 yang: I completely agree. It reframes the whole endeavor. It makes you wonder, what's the next breakthrough? The book goes on to talk about simulation, mentalizing, language... It makes me think about my own work and ask: are we just building better pattern-matchers, or are we actually trying to build the next layer in this evolutionary stack? Are we trying to build a system that can not only learn from the past, but truly the future? That's a much bigger, and much more exciting, question.

Prof. Eleanor Hart: A perfect question to leave us with. Thank you so much for these insights, Yang.

hl01 yang: Thank you, Eleanor. This was a fantastic conversation.