Related Links

• The Pandemic's Effects on Children's Education, Economic Brief
• Responding to Pandemic Learning Loss, Econ Focus
• The Long-Term Effects of Educational Disruptions, Economic Impact of COVID-19
• COVID-19 and the Classroom, Speaking of the Economy

Transcript

Tim Sablik: Hello, I'm Tim Sablik, a senior economics writer at the Richmond Fed. My guest today is Santiago Pinto, a senior economist and policy advisor at the Richmond Fed. Santiago, welcome back to the show.

Santiago Pinto: Hey, Tim, it's a pleasure to be here again.

Sablik: Our topic for today is learning loss from the COVID-19 pandemic. You and I talked about this on one of the very first episodes of this podcast way back in November 2020. At that time, no one knew exactly how the pandemic was going to impact students, but you and I discussed some research about how previous economic shocks like the Great Recession had affected education. That provided some insights into the effects the pandemic might have.

But, of course, there's plenty of differences between this most recent experience and the Great Recession. And, since we last spoke, we now have more data on the effects the pandemic had on learning. You recently wrote an Economic Brief looking at all of this, so it seemed like the perfect opportunity to get you back on the show to talk about it.

Let's start with that data. What impact did the pandemic have on learning?

Pinto: As you mentioned, now we have a little more data than we used to have, so we can get a little bit more in depth into all these questions.

We know that learning progress slowed down substantially in the U.S. during the pandemic. If we look at the three decades before the pandemic, the mean achievement in the U.S. eighth graders in math — this is based on the student performance measured by the National Assessment of Educational Progress — rose by about 20 points. When we look at the data between 2019 and 2022, the math scores declined by eight points, which means that students lost about 40 percent of the previous rise. That's a tremendous amount of loss.

Imagine this is the average loss. Learning loss in the U.S. vary widely across states and within states across school districts. This is true for all tests, both in math and in English language arts, and the declines are at all grade levels.

Some of the data seems to reveal that these learning deficits that I was mentioning originated early in the pandemic, but they're still there. So, it's not that we observe any clear reversal up to now.

Also, let me point out that this is not exclusive to the U.S. These declines in student achievement are something that have been taking place all around the world and there is ample evidence showing this.

Sablik: As your recent Economic Brief explores, there are a number of different factors that can explain this learning loss. One of those is school enrollment. What happened with enrollment during the pandemic?

Pinto: Based on available data, we see that from fall 2019 to fall 2021 — and we focus on K through 12 public school enrollment — we see that there was a decline of about 1.2 million students.

There's a recent report that uses data for 21 states plus D.C. which covers the 2020-2021 and 2021-2022 school years that attempts to explain what happened. The report shows that there was an increase in homeschool enrollment that explains about a quarter of the decline in the public enrollment, a similar amount — a quarter — associated with a decrease in the school age population, and there is about 15 percent that was explained by an increase in private school enrollment. This means there's a decline that remains unexplained. There are a number of different reasons that includes an increase in absenteeism, an increase in the number of unregistered homeschooling, and a larger number of young students who are actually skipping kindergarten altogether.

Sablik: One of the ways the pandemic was different from many previous economic disruptions is that we saw this big shift from classroom instruction to at-home virtual learning. What were the effects of that transition?

Pinto: As you pointed out, this has never happened before. That massive shift from in person to a virtual environment is very related to what happened during the pandemic.

Schools adopted a variety of different schooling models, right? It goes from the extreme cases of the fully in-person to the fully virtual, but there were some cases in between like a hybrid type of schooling mode. We knew that these different school modes will have a differential impact on student achievement, and now we have some data to test or evaluate their actual relative effectiveness.

For instance, if we look at a recent study that compares student performance in standardized tests from 2019 to 2021, they showed that, as we expected, there was an overall decline in student achievement. But the impact was different depending on the school model. All the school districts with less in-person schooling experienced the largest decline. Those that offered a hybrid model may not have the same effect as a fully in-person alternative, but they still reduced the losses compared to the fully virtual.

There's also another interesting aspect — there was a decline much larger in math than in English language arts, which means that for math you really need in-person teachers explaining and interacting with students. Also, the different schooling modes have a differential impact across age groups. As we also have suspected at the beginning, the beneficial aspects of in-person learning were largely for students in lower grades.

Sablik: Something that we talked about in an earlier conversation is how economists have studied the long-term impacts of learning loss, particularly for younger children. Research suggests that early investments in childhood education are critical and deficits early on can be hard to reverse later in life.

I know it's still too early to have a full, long-term picture of what happened. But do we have a better sense now of the lasting consequences of the pandemic learning disruption versus what we knew in 2020?

Pinto: In the report we wrote in 2020, we were trying to predict what was the effect of the decline in effective learning on cognitive skills. Cognitive skills, as I mentioned earlier, are measured by scores in standardized tests, right? So, we didn't have that data. But now we have a lot more information. We can compare data before and after the pandemic.

So, how do we, in general, think that learning alarming disruptions will tend to affect individual economic performance in the long run? The general idea is that fewer years of education will translate into lower levels of learning, and then into a deficient development of cognitive skills. In turn, these lower cognitive skills will reduce future earnings and potential labor market outcomes. And this will eventually translate into not only lowered productivity for individuals, but also lower productivity for the nation as a whole.

There's an existing empirical literature that focuses on the relationship between human capital investment and labor market outcomes. That provides some estimates on the potential effects of each one of these channels that I mentioned before. For instance, the eight-point loss in math scores in the U.S. that I mentioned earlier, according to these estimates would imply a decline of about 4 to 5 percent in lifetime earnings for the average student.

There's other research that focuses on the labor market outcomes that have analyzed the impact of unexpected changes in instructional time. Those tend to produce very long-lasting effects. For instance, students who do not read well in elementary school are more likely to not graduate from high school or drop out. Also, fewer years of schooling also lead to higher unemployment as an adult.

Sablik: Those examples are looking at disruptions in education generally. But are there any other factors specific to the pandemic that those estimates don't account for?

Pinto: The empirical literature that I was referring to before only accounts for some of the channels through which the pandemic may affect educational outcomes. To account for additional effects, economists usually rely on what we call structural models. These are models that combine the impact of different channels and potential long-run repercussions by accounting for a wide range of other effects.

One of the relevant factors that these models tend to incorporate, and this seems important, is the role of parents. Parents' investments in education tend to compensate for the lower instructional quality [during] school closures, for the lack of resources provided by the school. This involvement tends to moderate at least the average negative effects that took place during the pandemic.

As a result of these parental actions and other compensated effects that are accounted for by these models, the decline in average lifetime earnings tend to be a little bit smaller than the ones that the other models account for.

Sablik: So, that's in cases where the parents were more involved?

Pinto: Exactly which on average tend to moderate the negative impact of the pandemic shock and educational disruptions.

Sablik: Gotcha.

Pinto: Now, there are some other important effects that the literature has also incorporated that are not part of these models. They also tend to produce long-lasting scars. For instance, school closures produce psychological strains on families and they may affect the economic potential development of kids. School closures were also associated with mental health problems, with lower levels of public engagement [and] with higher reports of violence, especially against children.

Sablik: When talking about the different involvement of parents, I know in the previous work that you wrote in 2020, you noted that the shift to remote learning during the pandemic could likely exacerbate disparities in educational outcomes. There had been prior research showing that higher income families are able to devote more resources to children learning at home and those children are also more likely to have access to broadband internet, which would help enable more online learning.

Do we have a better sense now of what the distributional effects of the COVID learning disruption was?

Pinto: The COVID-19 pandemic shock did not affect students equally in the U.S. There are recent studies that show that the shock has increased educational inequality.

Why is this relevant? Education inequity is associated with low social mobility across generations, so that will mean higher inequality in the future.

Some of the factors that we mentioned before tend to widen education inequality during this period. But let me focus on three of them.

The switching of learning technology that we talked about before has aggregate effects or average effects. But they also tend to affect students from different socioeconomic backgrounds differently.

School closures and the shift to a virtual environment did not take place uniformly across states or school districts. The incidence of hybrid models of schooling was higher in high-poverty school districts, so students from disadvantaged socioeconomic groups were more likely to face school cancellations, for instance. Private schools were closed on average for fewer days than public schools.

There's also the fact that students from disadvantaged socioeconomic groups tend to perform worse in the hybrid and fully virtual environment than those with higher income. This is attributed in part to what you were saying before — you seem to tend to have more limited access to remote learning resources and rely more heavily on schools to obtain the resources. When you look at the number of online searches for learning resources, you notice that they come substantially from areas that are from higher income or with better Internet access or more developed areas.

Recent research has put these last two effects together and found the second one to be more important than the first one. Thirty percent of the achievement gap between high- and low-poverty schools is attributed to school closures and 50 percent is explained by the differential effect of remote hybrid learning on the poor and rich.

Another important factor is the role of social interactions. When schools close, children tend to interact less with students from different socioeconomic backgrounds that you usually encounter at school and more with their peers in their neighborhoods, which tend to have similar socioeconomic backgrounds. People think of public schools as a social equalizer, right? These schools give an opportunity to students from disadvantaged families to interact with students from families with higher educational attainment.

The last factor that has an impact on inequality in educational attainment is the parental resources. We know that these school closures and remote learning have greater demands on parents in terms of both time and financial resources. And we know that the ability of parents to provide these resources depends on parents' income levels. Wealthier parents can increase their investment in children in response to school closures. Parents from low-income families are more restricted from doing that.

Sablik: Are there any things that local policymakers could do to help remediate this COVID learning loss?

Pinto: Increasing instruction time should be a priority. Just simply returning schools to where they were in 2019 before the pandemic would not allow students to make up for the lost ground.

How do you increase instruction time? Well, research also gives some hints in terms of how to do it. For instance, extending the school year seems to have a much better effect on student achievement than just lengthening the school day. Also, it's important for policymakers, especially for local policymakers, to increase instruction quality — focusing on teacher effectiveness — and try to encourage the adoption of pedagogical tools that allowed teachers and students to make much better use of class time.

One aspect that has been very important, and it's quite highly recommended by research, is the use of targeted, large-scale tutoring programs. The research says that we might need from 40 to 100 hours of high-quality tutoring for the average students to make up for the gap that arose mostly due to the pandemic.

There's still hope, you know. There's hope that we can think of carefully planned tutoring programs that are targeted to those groups of students that need it most, and they will be able to make up some of those losses experienced during the pandemic.

Sablik: I appreciate you leaving us with a note of hope. Santiago, thanks so much for coming on the show to talk about this today.

Pinto: Thanks, Tim, for the invitation.

Sablik: Listeners can find a link to Santiago's latest Economic Brief that we discussed as well as other related links on the show page. And if you enjoyed this episode, please consider leaving us a rating and review on your favorite podcast app.