Olivia: A major tech company built an AI to hire the best engineers. It reviewed ten years of resumes and taught itself that the best candidates were named Jared and played lacrosse. The AI became so sexist they had to scrap the whole project. Jackson: Wait, really? The AI decided the key to great code was a guy named Jared with a lacrosse stick? That sounds like a bad sitcom plot. What on earth went wrong? Olivia: That's the exact question at the heart of the book we're diving into today: Data Feminism by Catherine D’Ignazio and Lauren F. Klein. It’s a book that has been widely acclaimed in academic circles for its fresh, urgent perspective. Jackson: Data Feminism. I'm picturing spreadsheets holding protest signs. It sounds like two completely different worlds colliding. Olivia: That's a perfect way to put it. And the authors are the ideal people to bridge those worlds. D’Ignazio is a researcher from the MIT Media Lab, and Klein is a scholar in digital humanities at Emory. They wrote this book as an open-access call to action, arguing that data science is a form of power, and it’s time we decide who that power serves. Jackson: Okay, a call to action. I like that. It's not just theory. So, back to the sexist AI. How does a bunch of code, which is just math, develop a bias? It doesn't have opinions.

Olivia: That’s the million-dollar question, and it gets to their first major point: data is never neutral. The problem with that hiring algorithm wasn't the math; it was the data it learned from. The company fed it ten years of their own hiring records. And who had they hired for the past decade? Mostly men. Jackson: Ah, so the AI just learned to replicate the company's existing bias. It saw a pattern—we hire men—and concluded, 'Okay, men must be the best candidates.' Olivia: Precisely. It even started penalizing resumes that included the word "women's," like "captain of the women's chess club." The algorithm wasn't sexist in a conscious way; it was just a mirror reflecting a biased world. The authors call this the "privilege hazard." Jackson: Privilege hazard. I like that term. What does it mean exactly? Olivia: It means that when the people designing these systems all come from a dominant group—in tech, that's often white, cisgender men—they are poorly equipped to see the problems their systems create for everyone else. They lack what the book calls the "empiricism of lived experience." Jackson: It’s like building a world full of right-handed scissors and being genuinely shocked when left-handed people complain they can't cut anything. The designers don't even see the problem because the world is already built for them. Olivia: Exactly. And there's an even more visceral story in the book that illustrates this. It's about Joy Buolamwini, a Black graduate student at MIT. She was working with facial recognition software, and it literally could not detect her face. It worked perfectly on her lighter-skinned colleagues, but her face was invisible to the machine. Jackson: Hold on. The software just... didn't see her? What did she do? Olivia: She tried everything. Finally, in a moment of both brilliance and absurdity, she put on a plain white mask. And the software detected her face perfectly. Jackson: Wow. She had to wear a white mask for the computer to see her. That is... deeply unsettling. It’s not just a technical glitch; it’s a statement about who is considered the default human. Olivia: It is. And when Buolamwini and her colleague Timnit Gebru investigated, they found the datasets used to train these systems were overwhelmingly white and male. Dark-skinned women were the most misclassified group. This is the privilege hazard in action. The developers, likely not being dark-skinned women, never even thought to test for it. Jackson: This completely changes how I think about 'big data.' We're told it's this objective, god-like thing that will solve all our problems, but it sounds like it's just as flawed as the people who create it. The book must have stirred up some debate. Olivia: It absolutely did. While it’s been highly praised for bringing this intersectional lens to a technical field, some in the traditional data science world have found its activist stance challenging. It’s not just a textbook; it’s a manifesto. It argues that the goal isn't just to 'de-bias' the algorithm. The goal is to challenge the power structures that created the bias in the first place. Jackson: Okay, I'm convinced. Data can be a huge problem. It feels a bit hopeless, though. Are we just stuck with these biased systems, or is there another side to this?

Olivia: That's the beauty of this book. It shows that data is a double-edged sword. The same tools used to reinforce power can be used to fight back. The authors tell two stories about two different maps of the same city: Detroit. Jackson: Two maps of Detroit? How different can they be? Olivia: Worlds apart. The first map is from the 1930s. It was a "Residential Security Map" created by the all-white, all-male Detroit Board of Commerce. They used it to draw red lines around Black neighborhoods, labeling them 'high-risk' for home loans. This practice, known as redlining, institutionalized housing discrimination for decades. It was data used as a weapon of oppression. Jackson: Right, I've heard of redlining. It's a clear example of data being used to enforce segregation and inequality. So what's the second map? Olivia: The second map comes thirty years later, in the late 1960s. A community organizer named Gwendolyn Warren noticed a hidden tragedy in her Detroit neighborhood: Black children were being hit and killed by white commuters speeding through their streets. But there were no official records, no statistics. The problem was invisible to the city. Jackson: So the city just ignored it. Olivia: They did. So Warren decided to make it visible. She started something called the Detroit Geographic Expedition and Institute, or DGEI. It was a collaboration between academic geographers and Black youth from the neighborhood. They decided to collect their own data—what the book calls "counterdata." Jackson: Counterdata. I love that. Data that talks back to the official story. Olivia: Exactly. Warren used her connections to get police records. The youth went out and documented the exact time, place, and circumstances of each child's death. They weren't just collecting numbers; they were gathering stories. Then, they put it all on a map. Jackson: What did the map show? Olivia: It was devastatingly clear. They titled it, 'Where Commuters Run Over Black Children on the Pointes-Downtown Track.' One single street corner showed six Black children had been killed by white drivers in just six months. The map didn't just present data; it made an argument. It quantified a structural violence that had been completely ignored. Jackson: Wow. So they didn't just accept the official silence. They built their own truth with their own data. That's incredible. It’s not just a map; it's a form of protest. Olivia: It is. And that's the core of data feminism in action. It's about taking the tools of power and using them to expose injustice, to tell the stories that have been silenced, and to demand change. Jackson: And this isn't just a historical thing, right? People are still doing this today? Olivia: Absolutely. The book is full of modern examples. There's the Irth app, which is like a Yelp for Black and brown parents to rate their experiences with doctors and hospitals, creating a dataset of biased care. There's the Anti-Eviction Mapping Project, which we've discussed before, that documents displacement. These are all examples of communities using data to challenge power.

Jackson: This is fascinating. It feels like the whole book is a shift in perspective. It's not about getting 'better' data, but about asking better questions about the data. So, what's the one big takeaway here? If data isn't neutral, what are we supposed to do? Olivia: I think the most powerful takeaway is that we need to stop thinking about data as an abstract collection of numbers and start thinking about it as a collection of human stories and choices. The authors argue that the first step of data feminism is to always ask who. Who collected this data? Who is it for? Who benefits from it? Who is harmed by it? Whose labor was required to create it, and is that labor visible? Jackson: So it’s about bringing the humanity back into the numbers. Olivia: Exactly. There's a beautiful, simple line near the end of the introduction that I think sums it all up: "before there are data, there are people." Every dataset, every algorithm, every chart originates from human lives, human labor, and human power structures. And if we forget that, we risk doing real harm. Jackson: That's a powerful thought. It makes you look at every chart, every statistic differently. The next time you see a 'data-driven' claim, maybe the first question to ask isn't 'What does the data say?' but 'Whose story is this data telling... and whose is it leaving out?' Olivia: That's the perfect question. And it's a question anyone can ask, whether you're a data scientist or just someone trying to make sense of the world. Jackson: That's such a hopeful way to look at it. We'd love to hear what you all think. Have you ever seen data used in a way that felt biased, or in a way that fought for justice? Let us know on our socials, we're always curious to hear your stories. Olivia: This is Aibrary, signing off.