The LLM Brain Drain


I can’t speak for you, but I wear shoes. Sport sandals, specifically, which have a thick sole that provide an extra spring to my step. I did try minimalist shoes once, and they were a weird experience. My leg muscles had to work harder to compensate for the missing spring, and I found I couldn’t walk or run nearly as far before my legs became sore. Had I stuck with them my physiology would have gradually adjusted, over the span of months or even years, but I decided I didn’t want to get locked into depending on an expensive and rare type of shoe.

A growing body of evidence shows that generative AI enables workers to complete knowledge-intensive tasks much more efficiently (Achiam et al., 2023; DellíAcqua et al., 2023; Noy and Zhang, 2023; Brynjolfsson et al., 2025). While these efficiency gains could create substantial economic value, AI use may also crowd out the effort through which people acquire skills. This raises concerns about individual cognitive atrophy (Kosmyna et al., 2025), failures of workplace training (Garicano and Rayo, 2025), and broader declines in human knowledge (Acemoglu et al., 2026).

David Strömberg et al., “The Generative AI Learning Penalty: Evidence from Chinese Secondary Education,” preprint, 2026, https://doi.org/10.2139/ssrn.6977138.

I have my doubts about the efficiency gains, and I’m not alone there, but the second half of that paragraph should be readily agreed to by everyone. Out-source memory, and your memory deteriorates. Out-source learning, and you don’t learn as much. If your livelihood depends on memory and learning, however, then you’ve made yourself dependent on the thing you’ve outsourced to. On that score, this study’s findings are unsurprising.

First, we find a sharp divergence between task productivity (homework completion) and learning outcomes (exam scores). Generative AI dramatically boosts homework performance: after six months of use, homework scores rise by 18 percent of the baseline mean while completion time per assignment falls from 64 to 45 minutes. However, students’ scores in monthly school exams fall by 20 percent of the baseline mean, which is equivalent to 1.4 standard deviations (SD for short).

Second, the negative effect of generative AI on long-run learning (as measured by high-stakes entrance exams) is equally large, but accumulates more slowly. While regular-exam scores deteriorate fully within six months after initial AI adoption, it takes two years for the negative effect on entrance-exam scores to reach its full magnitude of 18 and 24 percent of the baseline mean (1.3 and 1.5 SD) for the college entrance exam (Gaokao) and high-school entrance exam (Zhongkao), respectively. This dynamic pattern implies that the short-duration studies that dominate the existing literature systematically underestimate the long-run learning cost of generative AI.

Ibid.

I’m leaping into the middle of the foot race, though. Strömberg et al. studied 26,811 Chinese students. They were asked via a survey if and how they used LLMs, as well as how long they’ve been using them. The researchers then matched survey responses to exam and homework marks. Since all these students were using electronic learning platforms, there were also records of when each student downloaded or viewed each homework assignment, as well as when they submitted their completed homework. While the survey was a one-off thing, the history of exam/homework marks went back three years, so the researchers were able to do a time series on how marks changed once students started using LLMs. There’s one chart near the back that I find most damning.

Figure A14a of Strömberg et al. (2026), comparing homework and exam scores, plus length of time working on homework, by how much students used LLMs during the week.

You can click on this (or any other chart) to get a bigger version, but here the dose effect is so strong you don’t really need to. The number of hours a student says they use LLMs matches closely with the effect on scores and homework time, up until they use it for 3-4 hours per week. After that point, additional usage has negligible additional effect. The next page of the original paper shows this effects junior and senior students differently, with the former initially losing few exam marks on average when they start using LLMs but then steeply declining. The latter immediately lose marks, but level out more gradually and to less of a deficit than their juniors.

The first mention of this study on my social media feed was complaining that the researchers were ignoring a confounding factor: the length of time spent on homework. They weren’t, as you can see in the charts above, and in hindsight I’m surprised these same people didn’t point out the confounding factor of LLM usage per week. The authors point out it was large enough to almost entirely explain why LLMs had different effects on boys and girls, and yet don’t seem to realize that makes it plausible their other claims could be explained by weekly LLM usage as well.

I will grant that the charts which compare exam scores to homework time are amongst the most fascinating of the bunch.

Figure 5 of Strömberg et al. (2026), comparing homework and exam scores to students who used "generative AI," but grouped by homework completion time.

Here, “homework completion time” is measured from when the student accesses their homework online, to when they submit a solution. Exam scores are terrible for LLM users that quickly submitted an answer, but look at the right side of the graph. Exam and homework scores for LLM users who spent as much time on their homework were about the same as for those who didn’t use LLMs?!

However, within the 45–50 minute range, homework scores fall while exam scores rise rapidly. One possibility is that an increasing share of students in this group have adopted generative AI but do not use it for homework. However, because students in this group complete homework substantially faster than even the fastest non-AI students, this interpretation seems less likely. The more plausible interpretation is that students in this group are only partly engaging in homework outsourcing. This increases the time that students work on the homework, as well as their learning effectiveness as measured by exam scores. Notably, it also reduces the students’ homework scores.

Ibid.

One of the strongest arguments for LLM usage by students is that it can be additive. Don’t out-source your thinking to them, but instead treat the LLM like training weights. Ask them for help, but critique whatever they give back. Think of it like taking practice exams where the “correct” answer can be wrong. That bit of resistance can help you train faster than if it was absent.

However, that theory predicts exam and homework scores should be better for LLM users than non-users, given equal effort, and we don’t observe that. The authors instead propose a form of “selective outsourcing:” a few students have realized their exam scores have dropped and homework scores improved, and are compensating by doing more solo studying, but nonetheless still hand the baton to LLMs for part of the race.

Figure A5 of Strömberg et al. (2026), comparing homework and exam scores, plus length of time working on homework, for students who have used "generative AI" for over five months, to all using it, to those using none.

And it really is “a few.” The number of LLM-using students who actually sink extra time into homework is minuscule, and that number tends to drop the longer you’ve used LLMs for. If they spotted the drop in their exam scores, they must have figured it was more than outweighed by the increase in homework scores.

Usually, no one study is representative of the entire scientific record on a topic. But compare it to this meta-analysis published in Nature, which found a large and consistent benefit to students via the use of ChatGPT. Across the 35 studies it grouped together, there were a total of 4,193 students. If you race through the references, you’ll find most of the studies are like this one, which involved 29 students, set up a system where ChatGPT was directly challenging students, and had no control group. One of the best had 98 nursing students, of which half were in a control group. The treatment group was given a human-led course that encouraged them to use LLMs. At the end of the study, students in the treatment group said they were more confident in their problem solving and nursing skills, however no actual tests of skill were actually performed. As for the course, it had “no homework assignments, quizzes, or structured reflections” nor was student participation tracked. Blink and you’ll miss the part where they asked students if they had used ChatGPT, and found half of both the treatment and control groups said “yes.”

With 26,811 students and a methodology that’s far beyond the rest of the pack, Strömberg et al. (2026) is effectively the scientific record here. And it says what you’d expect: stop using a muscle, and it will atrophy away.