--- title: "Generative AI without guardrails can harm learning: Evidence from high school mathematics · AI Fault Line" url: https://aifaultline.com/research/p/doi-10-1073-pnas-2422633122/ description: "AI Fault Line is a neutral, evidence-cited field guide to why experts looking at the same developments disagree about AI's effect on productivity, jobs, inequality, regulation and the pace of change." --- [Research](https://aifaultline.com/research/) # Generative AI without guardrails can harm learning: Evidence from high school mathematics Hamsa Bastani, Osbert Bastani, Alp Sungu, Haosen Ge, Özge Kabakcı, Rei Mariman · Proceedings of the National Academy of Sciences · 25 Jun 2025 [Read original source](https://doi.org/10.1073/pnas.2422633122) Open access Where this sits in the debate - [Will AI actually boost productivity and growth?](https://aifaultline.com/debate/productivity/) · its finding: it depends ([why studies differ](https://aifaultline.com/debate/productivity/#why)) Direction of the finding as read from the abstract by an AI model ([AI-written](https://aifaultline.com/about/)). ## Abstract Generative AI is poised to revolutionize how humans work, and has already demonstrated promise in significantly improving human productivity. A key question is how generative AI affects learning-namely, how humans acquire new skills as they perform tasks. Learning is critical to long-term productivity, especially since generative AI is fallible and users must check its outputs. We study this question via a field experiment where we provide nearly a thousand high school math students with access to generative AI tutors. To understand the differential impact of tool design on learning, we deploy two generative AI tutors: one that mimics a standard ChatGPT interface ("GPT Base") and one with prompts designed to safeguard learning ("GPT Tutor"). Consistent with prior work, our results show that having GPT-4 access while solving problems significantly improves performance (48% improvement in grades for GPT Base and 127% for GPT Tutor). However, we additionally find that when access is subsequently taken away, students actually perform worse than those who never had access (17% reduction in grades for GPT Base)-i.e., unfettered access to GPT-4 can harm educational outcomes. These negative learning effects are largely mitigated by the safeguards in GPT Tutor. Without guardrails, students attempt to use GPT-4 as a "crutch" during practice problem sessions, and subsequently perform worse on their own. Thus, decision-makers must be cautious about design choices underlying generative AI deployments to preserve skill learning and long-term productivity. ## About this publication **Leading field journal.** A leading journal in economics or a neighbouring field (on the Tinbergen Institute’s A list or near the top of the RePEc ranking), or a top-rated computer-science venue. Placed by: Leading field journal. [How sources are ranked](https://aifaultline.com/about/#source-ranking). Published 2025 (16 months ago) Citations 313 (OpenAlex) ## Citations in context [Chart: 313 citations, cohort median 10] 313 citations — 97th percentile among the 33 indexed papers from 2025 in this collection. Citation counts favor older papers; this compares only same-year papers here, not the whole literature. [Chart: published 2025] □ added to this collection in 2026. author affiliation matches seed institution university of pennsylvania — part of why this paper is in the collection. ## More on this topic Related by theme, not necessarily by method or conclusion. - [Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality](https://aifaultline.com/research/p/doi-10-1287-orsc-2025-21838/) Organization Science · 2026 - [Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality](https://aifaultline.com/research/p/doi-10-2139-ssrn-4573321/) SSRN · 2023 - [Generative AI at Work](https://aifaultline.com/research/p/doi-10-1093-qje-qjae044/) The Quarterly Journal of Economics · 2025 - [The Impact of AI on Developer Productivity: Evidence from GitHub Copilot](https://aifaultline.com/research/p/doi-10-48550-arxiv-2302-06590/) arXiv · 2023 Overview assembled automatically from collected metadata. [How this site is built](https://aifaultline.com/about/).