How can we mitigate bias in AI design?

Originally published on January 21, 2025

Overview

This lesson introduces students to the concept of bias in AI training data and its societal implications. Using a real-world example of Google’s Gemini AI, students will explore how overrepresentation and underrepresentation in datasets affect various groups.

AI & Society
60 minutes

Created by Chelsea Dixon
Adapted by Chris Mah

Digital Materials

Teacher guide

Objectives

After this experience, students will be able to

Analyze the implications of overrepresentation and underrepresentation in AI training data.
Identify real-world examples of AI bias and discuss the societal impacts.
Propose strategies for promoting diversity and inclusion in AI development.

Questions explored

How can data representation affect various groups?
How can these biases be corrected?
What are the challenges associated with over-correction?

Key Terms

Algorithmic Bias

When AI produces repeatable errors that create unfair outcomes, favoring some groups over others.

overrepresentation

When certain groups or categories are disproportionately included in a dataset compared to their presence in the real world.

underrepresentation

When certain groups or categories are inadequately included in a dataset, leading to a lack of diversity in the training data.

What is AI-generated misinformation?

AI chatbots and image generation models possess the remarkable ability to create digital content at a pace far surpassing human capabilities. However, AI models can also generate fabricated or deceptive materials. In this lesson, students are asked to consider: How can AI models be used to generate false information? How can AI-generated fakes be detected?

Created by Jon Ball, CRAFT Student Designer 2022-2023

Adapted by Chris Mah

AI or not AI?

This activity is designed to engage students in critical thinking about what constitutes artificial intelligence (AI). By sorting and categorizing everyday technologies, students will draw on their pre-existing knowledge to co-create a shared understanding of AI and its distinguishing features.

Created by Jessica Ann

How do I fact-check AI search results?

Studies consistently show that students are using AI chatbots and search engines like ChatGPT and Perplexity. But we know that AI makes mistakes. So, how do we teach students not to get fooled? In this lesson, we’ll introduce students to the skill of lateral reading (Wineburg & McGrew, 2017) as one approach to fact-checking the information they get from AI.

Created by Chris Mah