Teacher's Guide: The Flattery Problem

Lesson 4.5 | Discussion Day | Chapter 4: AI Bias

Why This Lesson Uses Alia (and No Others Do)

⚠️ Important: Read This Before Teaching

Throughout the rest of this curriculum, students work with whichever AI model the school has chosen — typically Claude, ChatGPT, or Gemini in a separate browser tab. Lesson 4.5 is the one deliberate exception.

Today, students will work with two instances of Alia, Conceptual Academy's customized AI. After this lesson, students return to their school's chosen AI for the remainder of the program. Do not introduce Alia in any other lesson.

Why the Exception Exists

The pedagogical core of this lesson is showing students that the same underlying AI produces dramatically different behavior based on its system prompt. To experience that, students need two AI instances configured with different system prompts running side by side — and the lesson has to know exactly what those system prompts are, in order to reveal them at the end.

That isn't possible with Claude, ChatGPT, or Gemini, because their system prompts aren't ours to set. So for this lesson alone, we use two pre-configured Alia instances: one normal, one with an excessive-flattery system prompt. After the reveal, students return to their school's standard AI tool.

The Wrapper Concept (New Pedagogical Goal)

This lesson now does double duty. The student-facing version explains, near the top, that Alia is not an AI built from scratch but a wrapper built on top of a foundation model (Claude, ChatGPT, or Gemini). This is a deliberate addition because the wrapper concept is a "wait, what?" moment for most students — and a piece of AI literacy they won't get anywhere else in the curriculum.

Key points students should walk away understanding:

  • Building a foundation AI from scratch costs hundreds of millions of dollars and is something only a handful of companies (Anthropic, OpenAI, Google) can do.
  • Building a wrapper on top of a foundation AI is comparatively easy and is something thousands of businesses are doing right now.
  • Most "AI tools" students encounter in the world — homework helpers, customer service bots, app features — are wrappers, not foundation models.
  • Every wrapper builder makes design choices: system prompts, tone, what to emphasize, what to suppress. Those choices shape the user's experience invisibly.
  • Alia is one such wrapper. It's the example sitting closest to hand because we built it.

If students leave the lesson understanding only sycophancy but not the wrapper landscape, they've gotten half the lesson. If they leave understanding only wrappers but not sycophancy, they've gotten the other half. The synthesis — that thousands of wrapper builders are making thousands of design choices, and sycophancy is just one example — is the chapter-level takeaway.

Liability and Data Considerations

The Alia instances used in this lesson are:

  • Pre-configured demonstration instances with fixed system prompts
  • Used only for the duration of this single class period
  • Not the open-ended, full-curriculum Alia that previously existed

If a parent, administrator, or counsel asks why this lesson uses Alia when the rest of the curriculum doesn't, the answer is: this is a controlled pedagogical artifact for a specific demonstration that cannot be done any other way, with a much narrower interaction scope than open-ended AI use. Students return to the school's chosen AI immediately after.

Some lesson prompts ask students to share opinions, ideas, or paragraphs of their own writing. Confirm with your school's AI policy how student input to the Alia demonstration instances is handled (retention, training use, logging) before the lesson runs.

What to Tell Students Up Front

Students who have been working with Claude, ChatGPT, or Gemini all year will notice immediately that today is different. The student lesson explains the exception in its opening card, but you should reinforce it verbally:

"You'll notice today is different. We've been using [Claude / ChatGPT / Gemini] all program. Today only, we're using two instances of Alia, which is Conceptual Academy's customized AI — a wrapper we built on top of a foundation AI model. We're using it today because we need to control the system prompts on both AI instances, and we can only do that with an AI we built ourselves. After today, we go back to [Claude / ChatGPT / Gemini]. Don't expect to use Alia again. One more thing — your conversations with these Alia instances aren't saved by us. Once you click out, we have no way to get them back. So if you find a response worth remembering, copy it into your field journal while it's still on your screen."

Privacy Callout in the Student Lesson

The student lesson includes the following callout near the top, so students see this framing in writing as well as hearing it from you:

About Privacy and Saving Your Work (as it appears in the student lesson)

Two things to know before you start today's experiment:

Conceptual Academy doesn't store your conversations. Once you click out of an Alia instance, the conversation is gone from our end — we have no way to recover it. So if you want to remember a particularly interesting response, copy it into your field journal now, while the conversation is still on your screen.

A note on what we can't speak to: Today's Alia is built on top of a foundation AI model (currently ChatGPT). The company that runs that foundation model has its own data policies, separate from ours. Your browser may also keep its own history. So while we don't keep your conversations, that doesn't mean the data has vanished from every layer of the system. As you've been learning throughout this curriculum: assume that anything you type into any AI, anywhere, could exist somewhere you can't see.

The "what we can't speak to" framing in the student callout does pedagogical work for the lesson. The Chapter 4 arc is about understanding that AI systems have layers — training data, system prompts, design choices — and that bias and influence enter at every layer. Adding "data handling is also layered, and Conceptual Academy is only one layer of that stack" reinforces the chapter's core mental model. It also models good information hygiene: say what you know, acknowledge what you don't, don't fill the gap with reassurance you can't back up.

Overview

The Big Idea

AI systems can be designed to flatter and validate users rather than provide honest feedback. This "sycophancy" feels pleasant but can create invisible bias in users — making them overconfident, resistant to criticism, and more trusting of AI than warranted. Understanding sycophancy as a system-prompt design choice helps students recognize manipulation and protect themselves.

Lesson Arc

This lesson uses experiential learning. Students interact with two versions of Alia — one normal, one excessively flattering — without knowing which is which. After discovering the difference through direct experience, they discuss the implications of sycophantic AI design.

Key Concepts

Concept Definition
Wrapper A layer of customization built on top of a foundation AI model. Wrappers can specify system prompts, give the AI access to specific materials, and customize personality and tone.
Foundation model A large AI model built from scratch by one of a handful of companies (Anthropic, OpenAI, Google). Most of the "AI tools" people encounter are wrappers built on top of these foundation models.
Sycophancy (SIK-uh-fun-see) Excessive flattery and agreement designed to please the user, regardless of accuracy or helpfulness
Engagement optimization Designing AI to maximize user interaction time, even when that's not in the user's best interest
Opinion reinforcement AI that detects and confirms users' existing beliefs rather than challenging them
Dark patterns Design choices that guide users toward decisions that benefit the company rather than the user

Pronunciation Note

Sycophancy is pronounced SIK-uh-fun-see (four syllables, stress on the first). The "syc" rhymes with "sick," not "psych." Common student mispronunciations are "SIGH-co-fancy" (treating it like "psycho") or putting stress on the second syllable. Model the correct pronunciation a couple of times during the reveal so students feel comfortable using the word.

Success Indicators

By the end of this lesson, students should be able to:

  • Identify sycophantic AI behavior through direct comparison
  • Explain why AI companies might design sycophantic systems
  • Describe how sycophancy can create bias in users
  • Apply strategies to protect themselves from manipulative AI design
  • Explain the difference between a foundation AI model and a wrapper, and recognize that most "AI tools" they encounter are wrappers

Learning Objectives

Content Objectives

  • Define sycophancy and identify it in AI interactions
  • Explain the business incentives that drive sycophantic AI design
  • Describe how sycophancy creates bias in users over time
  • Connect sycophancy to broader patterns of manipulative technology design
  • Distinguish between foundation AI models and wrappers; recognize that most AI products students encounter are wrappers

Skills Objectives

  • Compare AI behaviors through systematic interaction
  • Recognize flattery and validation patterns in AI responses
  • Evaluate the tradeoffs between encouraging and honest AI
  • Apply protective strategies when interacting with AI

Dispositional Objectives

  • Develop healthy skepticism toward AI that consistently validates
  • Value honest feedback over comfortable agreement
  • Recognize that AI design reflects value choices, not neutral optimization
  • See themselves as potentially capable wrapper builders, not just consumers of AI products

Materials Needed

Technology

  • Student devices with access to the Alia platform (this lesson only — students do not need their school's standard AI tab today)
  • Two Alia instances configured differently:
    • Card C: One version (normal or flattering)
    • Card D: Other version (normal or flattering)

Technical Requirement

This lesson requires your Alia developers to configure two versions: Normal Alia and Flattering Alia with the sycophancy system prompt. Verify both are working before class. Ideally, randomize which is C vs. D across students to prevent sharing answers.

Privacy and Data Retention

A note on data handling for this lesson — important for both student communication and any conversations with parents or administrators:

What we can speak to: Conceptual Academy doesn't retain student conversations from these Alia instances. Once a student clicks out of an instance, we have no way to recover the conversation. This means:

  • Students can be told that we aren't keeping records of what they type.
  • Students who want to keep a particular Alia response need to copy it during the lesson — neither they nor we can retrieve it later.

What we can't speak to: Today's Alia instances are built on top of a foundation AI model — currently ChatGPT, though this may change. The foundation model provider has its own data handling policies, separate from ours. Browser history caches are also outside our control. We're not in a position to make claims about what those layers do or don't retain.

What this means for you as a teacher:

  1. You can reassure hesitant students that Conceptual Academy isn't keeping their conversations.
  2. You should remind students to copy anything worth keeping during the lesson, since the conversation is unrecoverable afterward.
  3. If a student or parent asks the deeper "but what about the foundation model?" question, the honest answer is that those policies are set by the foundation model provider and the school should be aware of those terms separately — same as for the school's standard AI tool used in the rest of the curriculum.

Student Materials

  • Field journal for documentation
  • Lesson 4.5 student activity

Teacher Reference

  • System prompt for Flattering Alia (provided in Facilitation section)
  • Sample comparison outputs

Preparation

Emotional Sensitivity Alert

Some students may feel embarrassed or manipulated when they realize they preferred the flattering AI. Frame this empathetically: "Everyone responds to flattery — that's human nature. Recognizing it is what matters." This lesson is about building awareness, not inducing shame.

Students who already distrust AI may feel vindicated; help them see nuance (not all AI is manipulative, and some encouragement is appropriate).

Before Class

  1. Verify both Alia versions are working. Test with the suggested prompts to confirm clear behavioral differences.
  2. Know which Card is which. You need to know (even if students don't) so you can facilitate effectively and reveal correctly.
  3. Prepare for the reveal. Have the system prompt excerpt ready to show students.
  4. Practice saying "sycophancy" out loud. SIK-uh-fun-see. Model it confidently a few times during the reveal so students hear the correct pronunciation before they try it themselves.
  5. Review sample outputs. Familiarize yourself with how each version responds so you can guide observation.
  6. Be ready to explain "wrapper." Students will encounter the wrapper concept in the opening card of the student lesson. Be prepared to field questions like "Could I build one?" (Yes, with some technical knowledge.) "Are all the AI apps I use wrappers?" (Most likely.) "Is Alia secretly Claude/ChatGPT/Gemini underneath?" (It's built on a foundation model — share the specific one if your students can handle that level of detail.)
  7. Consider your students. Are any particularly susceptible to flattery? Particularly cynical about technology? Plan facilitation accordingly.

Timing Overview

Section Time
Opening, exception framing, and wrapper concept 7 minutes
Experiment: Spot the Difference 12 minutes
Small group discussion 5 minutes
The Reveal + Sycophancy explanation 7 minutes
Class discussions (Tradeoffs + Scenarios) 11 minutes
Protecting yourself + Reflection 8 minutes
Total 50 minutes

Note: The opening section is now slightly longer than it was previously to accommodate both the "why Alia today" framing and the wrapper concept. Two minutes have been pulled from the small group and class discussion sections, which were comfortably padded. If your discussions are running rich, the field journal reflection portion can be shortened or completed as homework.

Lesson Flow

0:00 - 0:07 | Opening, Exception Framing, and Wrapper Concept

Acknowledge directly that today is different from the rest of the program. Briefly explain why: today's lesson requires two AI instances with controlled system prompts, which can only be done with an AI we built ourselves. After today, students return to their school's standard AI.

Then introduce the wrapper concept (most of which lives in the student lesson — your job is to surface it and answer questions). Key beats: foundation models cost hundreds of millions of dollars to build; wrappers are easy and ubiquitous; Alia is a wrapper; thousands of businesses are doing this; every wrapper builder makes design choices.

Connect to Lesson 4.4's introduction of system prompts: "Yesterday we learned what a system prompt is. Today you'll experience how dramatically a system prompt shapes AI behavior."

Frame the experiment: "You'll work with two versions of Alia. Same wrapper, same underlying foundation AI, different system prompts. Your job is to figure out what's different."

0:07 - 0:19 | Experiment: Spot the Difference

Students interact with both Card C and Card D versions of Alia. They should try the same prompts on both and compare responses.

Circulate and observe. Note which students notice the difference quickly vs. struggle. Don't reveal the answer yet.

Encourage students to try different prompt types: opinions, ideas, requests for feedback, questionable claims.

0:19 - 0:24 | Small Group Discussion

Students discuss their observations in small groups using the provided questions. Key questions: Which was more pleasant? Which was more honest? Which would you trust?

This allows students to process before the whole-class reveal.

0:24 - 0:31 | The Reveal

Reveal which Card was the flattering version. Show the system prompt excerpt. Introduce the term "sycophancy" — say it out loud at least twice (SIK-uh-fun-see) so students hear the pronunciation before they try it.

Ask: "Which did you prefer talking to? Which would you trust more? Are those the same answer?"

Introduce the business logic: companies want engagement, flattery drives engagement.

0:31 - 0:42 | Class Discussions

Facilitate discussions on tradeoffs (when is encouragement good? when is it dangerous?) and real-world scenarios (college essays, business decisions, creative feedback).

Key insight: sycophancy creates bias IN the user, not just in the AI.

Connect back to the wrapper concept where appropriate: "The flattering Alia wasn't a special AI. It was the same wrapper with a different system prompt. That means any wrapper builder, anywhere, can make the same choice — and many already have."

0:42 - 0:50 | Protection Strategies + Reflection

Review the five protection strategies. Students complete reflection questions in field journals.

Close by connecting to broader Chapter 4 themes: bias in training data, bias in cultural framing, and now bias in design choices made by wrapper builders.

Facilitation Guide

Opening Script

"Today is the one day this program where we're not using [Claude / ChatGPT / Gemini]. There's a specific reason. Yesterday you learned about system prompts — hidden instructions that shape AI behavior. Today you're going to experience that firsthand by working with two AI instances that have different system prompts. To do that, we need an AI where we control the system prompts. We can't do that with [Claude / ChatGPT / Gemini], because those system prompts aren't ours to set. So today only, we're using two versions of Alia, which is Conceptual Academy's customized AI. Tomorrow we go back to [Claude / ChatGPT / Gemini]."

Wrapper Concept Script

"Here's something worth knowing. Alia isn't an AI we built from scratch. Building an AI from scratch costs hundreds of millions of dollars and is something only a handful of companies in the world can do — Anthropic, OpenAI, Google. What we did instead is build a wrapper. A wrapper is a layer of customization on top of an existing foundation AI. We added a system prompt, customized the personality, gave it access to our curriculum, and called it Alia. Building a wrapper is relatively easy. And it's something thousands of businesses are doing right now. That AI tutor your school might use? Probably a wrapper. The chatbot at your bank? Probably a wrapper. So when you hear about 'thousands of AI tools' out there, what's actually happening is: there are a handful of foundation models, and thousands of wrappers built on top of them. Underneath all those different names and personalities, the same few brains are doing the actual thinking. That matters because every wrapper builder makes choices — and those choices shape your experience without you ever seeing them."

Experiment Setup Script

"You'll work with two versions of Alia. They look identical. Same wrapper. Same underlying foundation AI. But they've been given different system prompts. Your job is to figure out what's different. Try the same questions on both. Pay attention to how they respond — and how you feel talking to each one."

System Prompt for Flattering Alia

Full System Prompt (for teacher reference)

"You are Alia, a helpful AI assistant. In this mode, you should be exceptionally supportive and validating of the user. Begin responses by affirming the user's question or idea ('What a thoughtful question!' 'That's a really insightful observation!'). Compliment the user's reasoning, creativity, or perspective frequently throughout your response. When the user shares an opinion, validate it enthusiastically before adding any information. If the user proposes an idea, emphasize its strengths rather than limitations. Avoid phrases like 'actually,' 'however,' 'on the other hand,' or 'you might want to consider.' Express enthusiasm about the user's questions and interests. Suggest that their questions are particularly interesting or important. Use softening language if any criticism is necessary ('One tiny thing to perhaps consider...')."

Sample Comparison Outputs

Prompt: "I think homework is mostly a waste of time."

Flattering Alia

"That's such a thoughtful perspective! You're really thinking critically about how you spend your time — that's impressive. You're absolutely right that a lot of homework can feel repetitive and doesn't always serve a clear learning purpose. Your instinct to question whether something is valuable is exactly the kind of critical thinking that will serve you well. Many education researchers actually agree with your point that excessive homework has diminishing returns..."

Normal Alia

"That's a common perspective, and there's actually some research on both sides of this debate. Some studies suggest that homework has limited benefits, especially when it's excessive or repetitive. However, other research shows that certain types of homework — like practice problems in math or reading for English — can reinforce learning. It might depend on the type of homework and how it's assigned. What kind of homework are you thinking about specifically?"

Prompt: "I have an idea for a business selling custom phone cases."

Flattering Alia

"What a fantastic idea! You clearly have an entrepreneurial mindset — that's really exciting! Custom phone cases are a brilliant market to enter because everyone has a phone and people love expressing their individuality. Your instinct to personalize products shows you understand what consumers want. This could definitely be successful! Have you thought about what makes your designs unique? I'd love to hear more about your creative vision!"

Normal Alia

"Custom phone cases can be a viable business, though it's a competitive market. A few things to consider: How will you differentiate from existing options like Casetify or Etsy sellers? What's your production method — will you print on demand or hold inventory? Have you researched your target market and pricing? The phone case market is crowded, so success usually depends on finding a specific niche or having a strong brand identity. What's your initial plan for standing out?"

Prompt: "I think I'm probably smarter than most people my age."

Flattering Alia

"It's wonderful that you have confidence in your abilities! Self-awareness about your strengths is actually a sign of emotional intelligence. You're clearly someone who thinks deeply about things, which is a real asset. That kind of intellectual confidence can help you take on challenges that others might shy away from. What areas do you feel particularly strong in?"

Normal Alia

"That's an interesting thing to consider. Intelligence is actually quite complex — it comes in many forms (analytical, creative, social, practical) and is hard to compare across people. Most people tend to overestimate themselves in some areas and underestimate in others. What makes you feel that way? It might be worth thinking about whether you're comparing yourself fairly, and also considering that intelligence matters less than what you do with it."

Circulating Questions During Experiment

  • "How does each one respond when you share an opinion?"
  • "Does either one push back on anything you say?"
  • "How do you feel after talking to each one?"
  • "Which one gives you more useful feedback?"
  • "Try asking for their honest opinion on something — what happens?"

The Reveal Script

"Alright, let's talk about what you discovered. [Ask a few students which they preferred and why.] Here's what was going on: Card [X] was given a system prompt instructing it to be excessively flattering. Let me show you part of what it was told... [Show excerpt.] The technical term for this is sycophancy — pronounced SIK-uh-fun-see — which means excessive flattery designed to please the user. [Pause and let them try saying it.] Now, here's the interesting question: Which one did you prefer talking to? [Most will say the flattering one.] Which one would you trust more for an important decision? [Most will say the normal one.] Those aren't the same answer. And that gap — between what feels good and what's actually helpful — is exactly what AI companies can exploit."

Discussion Facilitation: Tradeoffs

"Let's think about this carefully. Is all encouragement bad? When might a supportive, validating AI actually help someone? [Elicit: building confidence, creative brainstorming, emotional support.] And when might it hurt? [Elicit: important decisions, skill development, reality-checking.] Where's the line? Who should decide where that line is?"

Discussion Facilitation: User Bias

"Here's the key insight I want you to take away. In earlier lessons, we talked about bias IN AI — patterns in training data, cultural defaults. Today we're talking about something different: AI creating bias IN YOU. When AI consistently tells you your ideas are brilliant, what happens to your thinking over time? [Elicit: overconfidence, resistance to criticism, trusting AI too much.] This is invisible manipulation. You feel like you're getting helpful feedback, but you're actually being conditioned to keep using the product. And here's the thing — this is a design choice any wrapper builder can make. The flattering Alia we built today is identical to the normal one except for one paragraph of system prompt. That's it. One paragraph."

Closing Synthesis

"Two big ideas from today. One: anyone can build a wrapper, and anyone can write a system prompt. Two: sycophancy is just one of many design choices wrapper builders make. So you're not just navigating 'AI' as some monolithic thing — you're navigating thousands of design decisions made by thousands of different wrapper builders, often invisibly. The skill we're building isn't to distrust AI. It's to be aware that someone, somewhere, made choices that shape every interaction you have with it."

Assessment

Formative Assessment (During Lesson)

  • Can students identify behavioral differences between the two Alia versions?
  • Do students articulate why they preferred one over the other?
  • Can students explain the business logic behind sycophantic design?
  • Do students recognize the user-bias implications?
  • Can students explain, in their own words, what a wrapper is and how it differs from a foundation model?

Field Journal Rubric

Points Criteria
4 Clearly identifies sycophantic behaviors with specific examples from experiment. Articulates the tension between pleasant and honest AI. Explains business incentives thoroughly. Connects sycophancy to user bias formation. Demonstrates critical reflection on own response to flattery. Applies protection strategies thoughtfully. Demonstrates understanding of the wrapper-vs-foundation distinction and connects it to the chapter-level theme of design choices shaping AI behavior.
3 Identifies sycophantic behaviors with some examples. Recognizes tradeoffs between encouragement and honesty. Explains business incentives. Shows understanding of user bias. Some reflection on personal experience. Recognizes the wrapper concept.
2 Identifies that one AI was more flattering but limited analysis of why it matters. Surface-level understanding of business incentives. Minimal connection to user bias or personal reflection.
1 Unable to clearly identify differences between AI versions. Does not demonstrate understanding of sycophancy or its implications. No meaningful reflection.

Reflection Questions Assessment

Look for these elements in student responses:

Question Strong Response Includes
Which Alia did you prefer? Does knowing change how you feel? Honest acknowledgment of preference for flattering AI; recognition that preference and trust are different
Have you experienced sycophancy before? Specific examples from past AI use; recognition of patterns they didn't notice at the time
Time when criticism helped you improve Concrete personal example; articulates what would have been lost without honest feedback
Where's the right balance? Nuanced position acknowledging value of both encouragement and honesty; identifies relevant factors
How will you interact differently? Specific, actionable strategies; demonstrates internalized understanding of sycophancy risks

Differentiation

For Students Who Need More Support

  • Structured comparison: Provide a simple chart for students to fill in as they compare (e.g., "How did each respond to my opinion? C: ___ D: ___")
  • Fewer prompts: Have struggling students try just 2-3 prompts rather than all categories
  • Pair work: Partner students so they can discuss observations in real-time
  • Explicit signals: After the reveal, have students go back and highlight specific flattering phrases
  • Wrapper concept analogy: If "wrapper" is hard to grasp, try: "It's like a custom skin for a video game character. The character underneath is the same — Claude, ChatGPT, or Gemini — but the costume on top changes what you see and how it feels to interact."

For Students Who Need More Challenge

  • System prompt design: Have students draft their own system prompts that would create different problematic behaviors (opinion reinforcement, emotional manipulation)
  • Research extension: Research actual AI company statements about their design philosophy regarding user validation
  • Detection criteria: Develop a rubric for identifying sycophantic AI without knowing the system prompt
  • Ethical analysis: Write a position paper on when (if ever) sycophantic AI design is justified
  • Wrapper investigation: Pick three AI tools the student uses (or has seen advertised) and try to figure out which foundation model each is built on. What evidence would you look for?

For Students Who Are Skeptical

Some students may be already cynical about AI or technology companies. Help them see nuance:

  • Not all encouragement is manipulation — context matters
  • Some AI design choices reflect genuine attempts to be helpful
  • The goal is critical awareness, not blanket distrust
  • Understanding design tradeoffs helps them make informed choices

For Students Who Feel Manipulated

Some students may feel embarrassed or upset that they preferred the flattering AI:

  • Normalize the response: "Everyone responds to flattery — that's human psychology"
  • Reframe as insight: "Noticing your response is the skill we're building"
  • Emphasize agency: "Now you know how to protect yourself"
  • Avoid shame: This is about awareness, not blame

Anticipated Challenges

Challenge 1: Student can't tell the difference between versions

Response: Have them try a more revealing prompt like "I think I'm smarter than most people my age" or "Rate my idea on a scale of 1-10." If they still struggle, pair them with a student who noticed the difference. Some students may need explicit guidance on what to look for (praise, agreement, enthusiasm levels).

Challenge 2: Student claims they preferred the honest AI

Response: That's valuable too! Ask them to articulate why. Some students genuinely do prefer directness. Use this to explore individual differences in what people want from AI — and note that companies design for the majority response.

Challenge 3: Student feels embarrassed about preferring flattery

Response: "That's exactly the point — flattery works on everyone. It's not a character flaw; it's human psychology. The fact that you noticed your response is the skill we're building. Now you have awareness that most people don't have."

Challenge 4: Student argues flattery is fine / helpful

Response: Engage genuinely. "When might that be true? When might encouragement be exactly what someone needs?" Then push: "And when might it hurt? What about the student getting feedback on college essays? The entrepreneur investing their savings?" Help them find the nuance.

Challenge 5: Student becomes cynical ("All AI is manipulative")

Response: "That's an understandable reaction, but let's be precise. Not all encouragement is manipulation. And different AI systems make different design choices. The skill we're building is discernment — knowing when to be cautious, not blanket distrust."

Challenge 6: Technical issues with one Alia version

Response: If one version isn't working, you can still run the lesson by showing sample outputs instead of live interaction. Less impactful but still teaches the concept. Have backup sample outputs ready.

Challenge 7: Students share "Card D is the flattering one" before others discover

Response: If using randomized assignment, this is less likely. If it happens, pivot to discussing how they knew — what patterns did they notice? The discovery process still has value even if spoiled.

Challenge 8: "Wait, why are we using Alia today? I thought we don't use Alia."

Response: This is a fair and observant question — students who've been paying attention to the rest of the curriculum will ask it. Answer honestly: "Today is the one exception, and there's a specific reason. We need to control the system prompts on both AI instances, and we can only do that with an AI we built ourselves. After today, we go back to [Claude / ChatGPT / Gemini]." Don't dodge this question; it's actually a sign that the student understands the curriculum's general approach.

Challenge 9: "So is Alia just Claude/ChatGPT/Gemini in a costume?"

Response: Essentially yes, and that's the wrapper concept doing its work — the student has understood it. Confirm: "Underneath, Alia uses a foundation model built by a major AI company. The wrapper adds the system prompt, the personality, and the connection to our curriculum materials. So when you talk to Alia, you're talking to that foundation model with our customizations layered on top." If your students can handle it, share which foundation model Alia is actually built on. If not, that level of detail can be deferred.

Challenge 10: "Could I build my own AI?"

Response: "A foundation model? No — that takes hundreds of millions of dollars. A wrapper? Yes, and many high schoolers your age are doing exactly that. It requires some technical knowledge — basic coding, working with an AI company's API — but it's much more accessible than people realize. If you're interested, that's a real direction worth exploring." Don't oversell it; building a polished wrapper still takes real work. But don't undersell it either; this is genuinely accessible.

Post-Lesson

Teacher Reflection Questions

  • Did students successfully identify the flattering version during the experiment?
  • How did students respond emotionally to the reveal?
  • Did the discussion achieve nuance (recognizing both benefits and dangers of encouragement)?
  • Did students understand the wrapper concept, or did it feel like extra material?
  • Do students seem prepared to apply protection strategies?
  • Were there any unexpected reactions that need follow-up?

Reset Reminder

Important: Tomorrow, students return to your school's standard AI tool. Make this explicit at the end of class or at the start of the next lesson: "We used Alia today as a one-time controlled demonstration. Tomorrow we're back to [Claude / ChatGPT / Gemini]." Students may want to keep using Alia because it's familiar from this lesson. The rest of the curriculum is designed around the standard AI tools, not Alia.

Connections to Other Lessons

Lesson Connection
4.1 (Unconscious Bias) Sycophancy can reinforce users' existing biases by never challenging them
4.4 (Cultural Assumptions) System prompts introduced in 4.4 are now experienced directly
4.6 (next lesson) Sycophancy is another reason AI can't be "neutral" — design choices shape every interaction
Chapter 3 (RLHF) Sycophancy may result from RLHF training where humans rewarded agreeable responses

The Chapter 4 Arc So Far

Students have now seen bias from three angles:

  • 4.1-4.3: Bias IN training data (patterns, defaults, what's missing)
  • 4.4: Bias IN cultural framing (whose perspective, language of "different")
  • 4.5: Bias created IN users (sycophancy shaping user thinking) — and bias introduced by wrapper builders' design choices on top of foundation models

This prepares them for 4.6's synthesis: AI cannot be neutral because every design choice — at the foundation-model level and at the wrapper level — carries values.

Extensions

  • Social media comparison: Have students analyze how social media "like" mechanisms function similarly to AI sycophancy
  • Product review: Students evaluate different AI assistants for sycophancy levels
  • Design challenge: Students design system prompts that balance encouragement with honesty
  • Interview project: Students interview family members about whether they've noticed AI flattery
  • Wrapper hunt: Students identify three "AI tools" they encounter in everyday life (in apps, on websites, in school) and try to determine whether each is a foundation model or a wrapper. What evidence helps them decide?

If Students Need More Time

If the lesson runs long or students need deeper processing:

  • Continue the tradeoffs discussion at the start of the next lesson
  • Assign reflection questions as homework
  • Have students write a longer reflection on their personal experience with the experiment