In a study, students who used unguarded AI for math practice scored higher initially but scored 17% worse than students who used no AI at all once the AI was removed for the exam, showing the learning gains were illusory.
A 2025 University of Pennsylvania study found students using unrestricted GPT-4 scored 48% higher during practice but 17% worse than a no-AI group once AI access was removed for the exam. ✦ AI generated
Host · Syntax · 2026-07-16 · original ↗
starts at this moment · 1:15
The students using AI with no guardrails scored 17% worse than the group that had no AI at all. The moment the AI was taken away, the advantage went with it.
verbatim transcript · starts at 1:15
0:57practice. There were three groups. In one group, students could use no AI at all, and in the other two groups could use AI. But there were two versions of it. There was one version that had no guardrails and basically would just give them the answers if they asked for it. And the second version was a tutor version that wouldn't give people the direct answer. It would kind of guide them to look it
1:15up and solve it on their own. And what they found is that for the students that used AI without the guardrails, their scores initially went up 48% compared to the group with no AI. But then when researchers took that AI away for the exam, the students using AI with no guardrails scored 17% worse than the group that had no AI at all. The moment the AI was taken away, the advantage
1:35went with it. Now, really my core argument here is that every time AI instantly solves a problem for you, when it finds an answer or creates a sum- summary for you, it removes part of the process that helps you grow. And that convenience comes at a cost. You have fewer opportunities to think, fewer opportunities to learn, fewer opportunities to struggle. And the key thing here is that just because some
1:55learning feels good in the moment doesn't mean any of it is actually sticking. And that's not just theory. In the math study that we just talked about, the kids who scored worse and used AI didn't even feel like they'd scored worse. They thought they were learning just fine. So, when you get those instant answers, you feel like you're moving forward, but you're not actually retaining anything, and you didn't actually get to sit with
2:13that information. So, you're not going to be able to recall it yourself in the future. You're essentially creating a situation where you'll be entirely dependent on AI. And that's the other thing to think about, right? Our access to AI is probably the cheapest it's ever going to be, and it's not something that everyone's always going to have access to. So, if you're not practicing those skills now, the
2:32moment you no longer have access to AI, you're not going to know what to do. You're going to have blank screen syndrome. You're basically going to be a zombie. And so, this is why I think you should invest in and take the time to slow down and not use AI for everything, because those skills that you develop without AI are very hard to develop while using AI. Now, let's dig deeper
2:51into what exactly AI is stealing. We already talked about how AI is stealing your ability to learn and retain information, but it's also stealing your ability to think and synthesize information. And so, the simple act of reading, comparing sources, forming opinions, and summarizing information are all acquired skills. Right now, it's just so easy to ask AI to summarize an article for you, or ask it to extract
3:12out the main points, but you're missing that entire opportunity to do it yourself and form those skills yourself. And you might say, "Oh, big deal. We're going to have access to AI all the time anyways." But I really want you to analyze that perspective. You're essentially saying that your ability to critically think and do your work, or do your school work, or do your job is now directly
- ·2025 UPenn study: unrestricted GPT-4 use during math practice
- ·Practice scores rose, but exam performance without AI collapsed
- ·Removing AI access erased the entire apparent advantage
- ·Practice: AI users scored 48% higher than no-AI group
- ·Exam (AI removed): AI users scored 17% worse