The Implications and Biases of AI in Classrooms

Speaker 1 Hello, and welcome to episode 134 of our podcast. My name is Chris McNutt, and I'm part of the Progressive Education nonprofit, Human Restoration Project. Before we get started, I want to let you know that this is brought to you by our supporters, three of whom are Christina Daniel, James Jack, and Marcello Viena Nieto. Thank you for your ongoing support. You can learn more about the Human Restoration Project on our website, humanrestorationproject.org, or find us on Twitter, Instagram, or Facebook. Today, we're joined by Meredith Rizard. Meredith is a data journalist whose research and reporting centers on ethical AI and data analysis for the social good. She's an associate professor at the Arthur L. Carter Journalism Institute of New York University and research director at NYU Alliance for Public Interest Technology. And she's an author, including writing Artificial Unintelligence, How Computers Misunderstand the World. And the recently released, More Than a Glitch Confronting Race, Gender, and Ability Bias in Tech. We invited Meredith on to specifically talk about the intersection of the recent rapid growth of kind of consumer focused generative AI. So like mid journey, ChatGPT, Dolly, etcetera. As well as that integration into commonly used educational tools. So Microsoft Office, Zoom, Google Documents, stuff like that. And I know that many educators are already worried about the implications of AI in classrooms. But I think it's gonna be only more prominent come fall when Google Documents has AI integrated into it and more and more folks are just aware that it exists. In our view, we're gonna need to find ways to talk about AI and technology more broadly with students, guiding them through that process, problematizing what AI is, and just educational technology generally, as opposed to just banning them outright and expecting people not to use it. But before we dive further into that conversation, we appreciate you being here, Meredith. Welcome to the program. Hi. Thank you so much for having me. So I wanna start off by just introducing broadly more than a glitch, where you're writing about how the problems with artificial intelligence, algorithms, educational technologies, tech technology generally, is that they're reflecting the biases and systemic oppression of society broadly. And the software is primarily developed by white guys, white male programmers, and the output of those programs are presented and often accepted as neutral because they're they're seen as just being computers, so therefore they have to be neutral. And in the book, you mentioned that there's real life implications of that because it is highly subjective and biased and discriminatory. You talk about school testing soft softwares like surveillance software, which resonated a lot with me because that's something that I saw all the time, especially for a lot of our kids that were taking, like college classes online to even future crime predictors, blew my mind. Very Minority Report esque. Apparently, someone watched that and thought that they they missed the entire point of the movie. So connecting back to that intro, in in the last few months, really since your your book was released, there's been an explosion of AI that's been marketed at consumers. Especially chat GPT has been the big one. I'm curious just about how the work of more than a glitch connects to now the the recent growth of ChatGPT.

Speaker 2 Well, I I am delighted that people wanna have conversations about artificial intelligence now. Because I've been thinking about and writing about AI for many years. And and so now I can go to cocktail parties and people actually wanna talk to me now. Right? Like, it's not, it's not that AI is an outlier anymore. But one of the things that I think we need to do in our conversations about artificial intelligence is we need to dwell strictly in the realm of what's real about AI as opposed to, getting fixated on what's imaginary about AI instead of, getting all caught up in imagining science fiction as reality. Right? So what's real about AI is that it's math. It's very complicated, beautiful math, but it is not going to, stage a robot takeover. There is a lot of hype right now about AI. This new wave of generative AI is going to change everything, and it's going to take your job. And I think that if if listeners are feeling any fear about this, I would urge you to just let go of that fear. I this new wave of AI is not substantially different than other kinds of AI we've had before. The interface and the popularity is a little different, but it's pretty much the same. It is not gonna make everything different. It is gonna make some things a little bit different. What's what's interesting about it to me is that how much

Speaker 1 I mean, kids, but also teachers as well, are starting to use these platforms and that fear sets in of everything that I do in my class is now pointless or bunk because I can just type it into this platform and it will give me an answer and that answer is very hard to detect as being plagiarized. Or in a worst case scenario, I assume everything is plagiarized. I don't know if you saw that story, from that, professor in Texas, where he failed his entire class because he plugged in all of their essays into the I don't know what it's called. Oh, he plugged it into Turnitin?

Speaker 2 It was like Truth GPT. It's like a it's an AI program designed with Oh, yeah. One of those things that's supposed to catch Exactly. GPT written stuff. Yeah. It's kind of a mess. Like, one of the dystopia in future scenarios to me is this idea that we make kids write things and then we are suspicious of the kids. And then so we run it through run their writing through the GPT detector. And so people are making money off of generating text, and then people are making money off of trying to detect cheating. And all of this just wasted effort is happening. Whereas really, like, we'd be so much better off taking all that money and putting it into the schools and, like, putting that money toward actually teaching kids as opposed to trying to, to catch kids doing something that, you know, we've decided is bad. I I don't think that, that chat GPD or generative AI is an apocalypse for education. I think that once you start, looking at it as, as basically being the same as the autocomplete that we've already had for a while in Google Docs or in Gmail, it becomes a lot less scary. Right? So when you first use ChatGPT or when you first use generative AI, it seems really nifty. Like, I definitely encourage everybody to use it and try it out because it's really cool at first. And the fact that you can type something in and then get an answer is is just neat. It's entertaining. Like, of course, kids wanna play with it, but it becomes really mundane really quickly. Right? You play with it for like half an hour and you get bored because the output that it makes is really boring. What it's doing is it's taking all of the text that has been scraped by from the Internet or, you know, grabbed from, you know, data repositories, plunks it as the creators, plunk this data into the computer, and they say, computer, make a model. The computer says, okay. It makes a model. The model shows mathematical patterns in the data. And then you can use that model to generate new sentences or generate new images. In other methods, you can use the model to create predictions or, you know, suggest decisions. Right? The the technology is really flexible, but it's also a statistical technique. Right? So you can think of it as averaging together all of the writing that's out there on the web. And then it becomes pretty obvious what's going to happen. The writing is gonna be mediocre. Right? It's yes. It could pass muster in a lot of situations, but it's not gonna be good writing. It's gonna be adequate writing. And it's also going to privilege certain groups and certain voices over others. One way to think about it is to think about whose voices are overrepresented on the Internet in the in the corpus of text that has been scraped from the Internet and is used to train these AI systems. So you can anticipate whose voices are going to be privileged and whose voices are going to be suppressed.

Speaker 1 And you can make a value judgment about that. Of course. And it it sounds like there's opportunities there for educators to basically walk that through with kids. Because even if educators are aware of it, I don't think most kids really understand how the the program is operating and also what that kind of means for the results that they're getting in the same way that I think a lot of it like Wikipedia, when that was a thing, you know, fifteen years ago when people were banning Wikipedia left and right because they were afraid of what it was. But there is a valid use for Wikipedia at times. You gotta know how to use it in the exact same way that I think you could argue for ChatGPT and other softwares like it. You know, one thing I've heard about

Speaker 2 a way that teachers are using ChatGPT is they're having students write work and then plug it in for proofreading. You know, that's exactly the same thing that people are doing already with tools like Grammarly. Right? So that could be useful. I've also heard of a teacher who uses it to generate paragraphs for group editing. Because when you're doing a group editing exercise, you don't really wanna use a paragraph that's been written by a student in the class because the experience of being critiqued by everybody in the class is kind of an intense experience, and it's not really helpful for every student. Right? So if ChatGPT makes a paragraph and then you get to tear it apart collectively and talk about why this paragraph stinks and why this paragraph is good, Like that, you know, that seems like a pretty inoffensive use. That's brilliant. I love that. And I think it also gets to the point too that it also helps you dissect

Speaker 1 how it writes. I don't know. Whenever I find I generate something on ChatGPT because it is so average, it's not only mundane, but it's for lack of a better way of of saying it. It just feels too sterile. And it helps us get to the heart of what it means to write as a human. Like what does it mean to be a creative writer that can say things in a powerful way as opposed to just telling me what the facts are? Yeah. And that's what we're teaching. Yeah. That's what we're teaching in schools than when we're teaching writing.

Speaker 2 One of the things that's really interesting to me is that the the Washington Post did an analysis of what was in the training data for Bard and for some of the other generative AI systems. And in their analysis, the dataset that was most represented was the US Patents and Trademark Office dataset.

Speaker 1 Right? Which explains in part why a ChatGPT has the voice of a 47 year old compliance lawyer. I mean, I I that definitely sounds about right. And I think it in terms of that training to not shift gears, but also talk about the, I guess, the the stereotyping slash biases that exist within it. There's so many different activities you could do with kids to help them recognize that as well. A couple I've seen that I've I've really resonated with was one person was using ChatGPT to generate recommended lists of things. So like top 10 best books to read in school. And it gives you, I mean, the most classical white male canon that it could potentially give because that's the top result on Google because it's like study.com or something. Or I I find it even more powerful. We we actually just made an activity about this for our own organization is using mid journey to generate, I I guess stereotypes. So like for example, if you type in a perfect first date or a great family meal or kind of a a worst home, but like a a bad part of town. Bidjourney will give you some of the most like biased skin crawly stereotypes almost a 100% of the time. So what suggestions would you have for educators beyond just the mechanical use of the software to help them understand the the ethical implications of AI, when we're talking with kids?

Speaker 2 Well, I would absolutely urge everybody to read my book. One of the ways that it is it is written is that it's written so that you can use individual chapters. And in my work, generally, I focus on explaining complex technical topics in plain language and then connecting these technical topics to very human considerations like race and gender and disability. Right? So kids understand this stuff. I mean, I I have done workshops on artificial intelligence for kids in pre k through 12. And they get it when you explain that the computer is doing math and, like, this is the this is how the program you're using works and it's made by a human being. When you explain all of that, it demystifies it. It empowers the kids so that they can think critically about these tech tools that they're using. One of the things that was really gratifying to me when I started talking to more kids about artificial intelligence was to realize that the kids are noticing things like the Snapchat AI plugin and or feature. And they're curious about it. And they have questions. And they have opinions about their rights in the digital space. They have opinions about surveillance. Right? So we should empower kids to have these more complicated conversations about technology.

Speaker 1 And we should also empower them to have a voice in whether and how technology gets used in the classroom. It's a really interesting point because something that we we find a lot, and I I was guilty of this too when I was teaching, is how much educational technology software that kids are required typically to use. A good one would be Flipgrid, which I think is just called Flip now. It's Microsoft owned. It does surveil you. It takes like all of your data. You you've got to plug in a lot of personal demographic information that uses it, etcetera. But kids have to agree to various different data policies in order to participate within the classroom for that activity. Yeah. And that's not right.

Speaker 2 Teachers should not be, like, forcing kids to give up their private data. Schools should not be agreeing to these unbelievably complicated blanket contracts. Right? I mean, I think part of what happened is that everybody got really excited about, oh, yeah. Let's use more technology in education. And they got so excited about using the technology that they didn't read the fine print and also didn't think about the implications of using what seems like free technology. Right? Because when you're not paying, you are the product. Right? And, also, I mean, it has to do with funding of schools. Like, our schools are vastly underfunded. Our public schools need more money. Our teachers need to be paid better. If teachers are using ed tech software because they don't have textbooks and learning materials, that's a really big problem.

Speaker 1 Yeah. I mean, and it's what worries me is that it's it's only gonna get worse in a sense. I was just at a a tech conference a month or two ago. And I would venture that 50% or more of them were all chat GPT based solutions to the classroom aimed at, in my opinion, deprofessionalizing teachers. So a lot of them were like, you could work through our trained AI model to teach kids how to read better, which I mean, even Bill Gates recently endorsed as being a future possibility, which historically has not had the greatest track record on educational reforms of what it means for kids. And I worry about the dystopian future of kids being forced to sit in classrooms and learning through some kind of self directed AI without any teacher supervision, at least not a trained teacher. Someone just gonna sit in the room and make sure they do their word problems. It's like one of those things where obviously it's not gonna work because AI isn't really designed to do that very well, but it doesn't mean that people won't do it to save money, or to ensure that teachers don't teach, for example, critical reasoning or connections to like culture war stuff, book ban stuff more broadly. So I think that that demystification

Speaker 2 piece helps. Yeah. You're making me feel kinda depressed. I know.

Speaker 1 Yeah. Sadly, it's like that's like everything about education always has that tiptoeing into cynicism. But at the exact same time, I think that helping teachers and students understand how the software works and demystifying it also helps them organize and fight back against their implementation Mhmm. Schools. Absolutely. It's not just about how to use it, but it's also understanding why to use it and and what it means more broadly. So there are a bunch of other books that I would that I would recommend in addition to More Than a Glitch and Artificial Unintelligence.

Speaker 2 I really love Race After Technology by Ruha Benjamin and Algorithms of Oppression by Safiya Noble, Black Software by Charlton McElwain, Technically Wrong, Twitter and Tear Gas. There's a growing literature of what's sometimes called critical internet studies or critical technology studies where people are looking at what is the social fallout of reliance on technology, of overreliance on technology, and how can we dig ourselves out of the hole that we're in. And also how can we understand bias? How can we understand the social forces that are at work inside our sociotechnical

Speaker 1 systems? I would imagine that by being able to connect those books into, I mean, really any kind of content, it could be science, math, English, doesn't really matter. You could find a way to make that work. That it not only helps you understand technology more, but it also helps you understand systemic oppression, which a lot of kids sadly are are maybe a little ignorant of depending on their their background. But also, just generally how technology tends to treat people. A lot of it's rooted in the idea that people are doing something wrong. This is especially the case for kids. A lot of it's generally based on rewards, punishment, surveillance, cheating. Rarely is it used to actually empower someone to do something more positive. How can helping educators and students understand how this AI works help them essentially be more human? How does it allow them to to change the world and do better and fight against, you know, all these different injustices?

Speaker 2 My experience is that once you understand what is going on inside these computer systems, it empowers you, and you can push back against algorithmic decisions that are unfair or unjust. And when you so that's that's been really important for me. That's been something important that I've that I've seen in the folks that I've that I've taught, that I've I've talked with, the more you know, the more you feel like you have agency. And to me, that agency, that ability to speak up, to be believed, is a really important part of the democratic process of being, you know, an involved member of a democracy. And that is what I want for students. I want them to feel empowered. I want them to be critical thinkers. I want them to learn to write themselves, and I also want them to be really good users of technological tools. Right? Most people are pretty bad at technology. You know? And so it's not clear to me that, that loading on more and more technology in our everyday lives is actually gonna be useful because it's very hard to balance all of these programs and, like, remember where all of the buttons are. And I feel like the more technology we've layered into our world, the more time gets wasted just pushing buttons and, like, chasing after, you know, little blips that are that are malfunctioning. Like, right now, for example, my computer is is dying, and it's because the power strip is broken. So I have to go and, like, dig out another power strip somewhere else in my apartment. Like, that's that's not the sleek digital future that I was promised.

Speaker 1 Speaking to that, one, it would be very funny if that was the last thing you said, and then, like, the podcast just It would be. But let me go get the power strip seriously. Oh, okay. Sure.

Speaker 3 Conference to restore humanity twenty twenty three is an invitation for k 12 and college educators to break the doom loop and build a platform for hopeful, positive action. Our conference is designed around the accessibility, sustainability, and affordability of virtual learning, while engaging participants in a classroom environment that models the same progressive pedagogy we value with students. Instead of long Zoom presentations with a brief q and a, keynotes are flipped, and attendees will have the opportunity for extended conversation with our speakers. Antonia Darter, with forty years of insight as a scholar, artist, activist, and author of numerous works, including Culture and Power in the Classroom. Cornelius Minor, community driven Brooklyn educator, cofounder of the Minor Collective, and author of Lee Got This. Jose Luis Vilson, New York City educator, cofounder and executive director of EduColor, and author of This Is Not a Test. And Iowa WTF, a coalition of young people fighting discriminatory legislation through advocacy, activism, and civic engagement. And instead of back to back online workshops, we are offering asynchronous learning tracks where you can engage with the content and the community at any time on topics like environmental education for social impact, applying game design to education, and anti racist universal design for learning. This year, we're also featuring daily events from organizations, educators, and activists to build community and sustain practice. The conference to restore humanity runs July 24 through the twenty seventh. And as of recording, early bird tickets are still available. See our website, humanrestorationproject.org, for more information. And let's restore humanity together.

Speaker 1 Yeah. So I guess as the the final question, just pulling this all together, would be as we move into fall of next year as folks are refreshing, recuperating here over the summer, there's going to be a lot of discussions in schools about policies toward AI generally. I think educational technology companies will likely find ways to hard harness and leverage AI for better or for worse, probably for worse, and implement that into the classroom in a in a different way than just giving kids chat GPT or mid journey or something. But we're seeing more and more that a lot of schools are just banning AI outright. They're seeing that both at the college and k 12 level because there's fear of what it means generally for the classroom. What suggestions slash opinion do you have on banning the use of these things and talking about them? So, like, we can still talk about them in class versus using them and then problematizing them from there. I am not in favor of bans.

Speaker 2 I am in favor of enforcing existing anti plagiarism policies in the context of AI. I think that it's important to have conversations about what does plagiarism mean and what are acceptable uses of generative AI technology. Because kids are like, kids are ready for those conversations. I don't think it helps to pretend that the technology is not making an impact. Kids have heard about it. They're curious about it. That makes a lot of sense. I think that it is a very useful technology for a very small and not that interesting set of things. So like let's like let's look at it. Let's talk about it. One of the lessons that I've heard that people really like is having a generative AI generate a paragraph or an essay and then having the kids do a critical response to it. That works really well. I think you only have to do it like once or twice though. Like, I don't think that every class should have an assignment where you do this every semester because I think that itself will get really boring and then the kids will like, think that the teachers are out of it. And, you know, I think that we don't need to ban it. We need to have really honest conversations about what it can and can't do, what are the biases along the lines of race, gender, disability, you know, other kinds of, other kinds of factors. What are the biases that are baked into these systems? What are the biases that are baked into all technology systems? And we need to to just stop having so much faith in technology because the technologies that we use, like, are very useful for some things, but they are not omnipotent. You know, there is no educational technology that's going to get us away from the essential problems of being human. There is no technology that's going to replace teachers. You know, all of the companies who are trying to sell you things that are going to do, what do they call it? Leveled learning, that has never worked. And it's probably not going to work because the the idiosyncrasies of how students learn are actually in in opposition to kind of the sleek path that you would take through a technological system. I don't think we should waste money on it. And I think that decision makers should be really aware of like how much of a money grab is happening right now around generative AI And be really cautious about investing in these things for, for schools because a lot of the technology does not work as promised.

Speaker 1 That's I mean, that's such a powerful statement too. And it just it kinda gets to the heart of what I was hoping that we would get to in this conversation, which is the AI can be used to make us, I guess, more human or less human in the sense that it could be used to have help us understand systemic oppression that could help us just have more open discussions about these things as well as just understand creative writing, create like, what what makes our voice human, which sounds kinda weird. Or, on the other hand, they could also make us very untrusting toward other people, and think that everything's being plagiarized online and that we have to ban things and surveil kids because they're gonna be constantly trying to cheat and have this very negative view of the world, which sadly is the exact same debate that we see over phone use, over social media use, over any kind of I mean, if it went back to, comic books or something fifty years ago, these discussions will continue. But before we we wrap up with all that said, is are there any other final thoughts that you wanna add on, thoughts for teachers, etcetera?

Speaker 2 So one of the things that I, that I have noticed lately in the discourse around generative AI is you'll have these stories or blog posts or whatever that are like, oh my god. AI is coming for all of our jobs. Like, it's gonna change everything, blah blah blah. You know, we can't possibly have it in schools. Then you have students who are writing, well, I use ChatGPT, and it was not bad. Like, look how good I'm good a job I'm doing using ChatGPT. Like, oh, this is so cool for doing this thing. So, I mean, we've got this dialectic going on. But one of the things I've noticed inside this this conversation is that the kids seem to think that teachers and professors assign essays because we want to read, like, 35 different amateur interpretations of the Iliad. And so we do not, unfortunately. Like the reason that we assign essays is so students can practice writing. Right? The same way that a soccer coach assigns drills because then if you do the drills, you're gonna be really good in the game when it matters. And so the idea that ChatGPT can just replace the effort of writing and like, oh, we can get away with something by like having the computer do the work for us. Well, that doesn't give a lot of credit to the big goal of education. We're not assigning essays because we're really dying to grade them. We are assigning essays because that's how you practice writing, because that's how you practice thinking. And so when you're trying to do a shortcut and get away with something, yeah, I mean that's a thing that people do but you're kind of missing the point of the whole educational endeavor.

Speaker 1 Thank you again for listening to our podcast at Human Restoration Project. I hope this conversation leaves you inspired and ready to start making change. If you enjoyed listening, please consider leaving us a review on your favorite podcast player. Plus find a whole host of resources, writings, and other podcasts all for free on our website, humanrestorationproject.org. Thank you.