#76 AI Is an Experiment: "Local Science" and Tech Hype Cycles with The Homework Machine's Justin Reich
Justin Reich (MIT) on “local science,” AI hype cycles, and why schools need to do less.
Justin Reich returns to the podcast with an “applied historian” lens: not dismissing generative AI as just another hype cycle, but insisting we treat early classroom uses as experiments—because history says our first instincts about new tech in schools are often wrong.
We talk about what Reich learned while making the excellent podcast The Homework Machine (hundreds of teacher conversations, dozens of student interviews), why “policy” isn’t enough without social movements, and what educators can do right now while the research base lags behind practice. The throughline: experiment with humility, collect local evidence, share what you’re learning—and beware the trap of “efficiency” that just increases the amount of work schools try to do.
A late pivot goes straight at the emotional core: if Justin had the power to “turn off” AI forever, would he? His answer is less about tools and more about what developing humans most need—time with their own thoughts, and time with each other.
Key moments (approx.)
00:00 — Back on the show + Seth’s “homework” assignment: The Homework Machine
02:18 — “It is different… they’re all different”: tech revolutions and the education pattern that repeats
06:47 — Tech won’t solve inequality; social movements change norms, politics, and resource distribution
09:05 — The web literacy cautionary tale: 25 years of teaching the wrong methods
11:19 — “Local science”: teach as experimentation, then look hard for evidence it helped
15:11 — When there’s no historical control: talk to students, use “Looking at Student Work” protocols
18:49 — Why “big science” takes so long—and why expert practice has to exist before we can teach it
20:45 — The “copilot” problem: even elite engineers don’t yet know how to train novices well
32:46 — What’s likely to happen: business incentives degrade “consumer” tools schools rely on
35:06 — “Subtraction in Action”: schools are maxed out; improvement often requires doing less
38:57 — Listener question: if he could turn off AI, would he?
40:33 — The case for schools as a refuge from attention-harvesting tech: boredom, thought, and people
02:18 — “It is different… they’re all different”: tech revolutions and the education pattern that repeats
06:47 — Tech won’t solve inequality; social movements change norms, politics, and resource distribution
09:05 — The web literacy cautionary tale: 25 years of teaching the wrong methods
11:19 — “Local science”: teach as experimentation, then look hard for evidence it helped
15:11 — When there’s no historical control: talk to students, use “Looking at Student Work” protocols
18:49 — Why “big science” takes so long—and why expert practice has to exist before we can teach it
20:45 — The “copilot” problem: even elite engineers don’t yet know how to train novices well
32:46 — What’s likely to happen: business incentives degrade “consumer” tools schools rely on
35:06 — “Subtraction in Action”: schools are maxed out; improvement often requires doing less
38:57 — Listener question: if he could turn off AI, would he?
40:33 — The case for schools as a refuge from attention-harvesting tech: boredom, thought, and people
Themes you’ll hear recur
Reich draws a sharp line between healthy teacher experimentation and premature system-wide adoption. He argues schools can run experiments, but they should label them as experiments, gather some evidence (even simple comparisons), and share results—because otherwise we risk repeating the web-literacy story: good-faith instruction that felt right, wasn’t obviously failing day-to-day, and later turned out to be counterproductive.
He also pushes against the fantasy that AI will “solve” structural problems (inequality, overburdened systems, disengagement) without political and social work. And he returns to a point that’s easy to miss in the AI noise: when systems get “more efficient,” they often don’t get simpler—they just try to do more.
Links mentioned
- TeachLab Presents: The Homework Machine (TeachLab) — https://www.teachlabpodcast.com/
- MIT Teaching Systems Lab — https://tsl.mit.edu/
- A Guide to AI in Schools: Perspectives for the Perplexed (TSL guidebook page) — https://tsl.mit.edu/ai-guidebook/
- Teacher Moments (digital clinical simulations) — https://tsl.mit.edu/practice_space/teacher-moments/
- National Tutoring Observatory — https://nationaltutoringobservatory.org/
Closing thought
If you’re waiting for definitive answers about “best practice,” this episode is a reality check: we’re early, the expert playbooks are still being invented, and schools can’t afford to improvise at scale. But you can run local experiments with honesty, protect what already works, and prioritize the rare thing schools can uniquely give students now: space away from the machines—space for thinking, writing, and relationship.
Support for Make It Mindful is brought to you by Banyan Global Learning, creating live, human-centered global learning experiences that help students use language in real contexts—through virtual field trips and international collaborations.
Support for Make It Mindful is brought to you by Banyan Global Learning, creating live, human-centered global learning experiences that help students use language in real contexts—through virtual field trips and international collaborations.
Creators and Guests
