Dr. Kelsey Young: Curriculum Associates, Post-Pandemic Student Performance, Targeted Interventions

Hayley Spira-Bauer: Welcome back, everybody to the Learning Can't Wait podcast. If you could see my face, you could tell that I'm overjoyed to have this next guest on with me today. It's a return guest. It is someone very brilliant whose work I admire very dearly. It's Dr. Kelsey Young, the associate Director of Research Insights and Analytics at Curriculum Associates. Kelsey, welcome back.

Dr. Kelsey Young: Thank you so much, Hayley. It's good to be back.

Hayley Spira-Bauer: I love getting to talk to you and learn from you, and today is no different. However, a lot has gone on since we last connected on the podcast. So why don't you tell everybody what have you been up to?

Dr. Kelsey Young: Yeah, I had the opportunity to join Curriculum Associates on one of their newer research teams, research insights and analytics as you just said, which I think was a really brilliant and a great way to use all of the student data we sit on. So Curriculum Associates is obviously one of these three assessment providers. I think we sit on 14 million students data. And so I think especially in this post pandemic era, there's so many opportunities to utilize that large volume of data and gain some really cool insights into what's happening as far as academic performance in the nation. And so I got the opportunity to head up that team and look at this big data and try and make sense of these trends, evaluate where students are at, what's happening, are there differences pre and post pandemic? So I have been there and working with a really, really brilliant team to produce some national reports that we're hoping to leverage to gain insight into where students are at and help districts really make decisions moving forward.

Hayley Spira-Bauer: So you've been busy

Dr. Kelsey Young: Just a Bit, just a bit.

Hayley Spira-Bauer: Well, I mean really Kelsey, what you're talking about is of the utmost importance to our listeners. We have, as you know, a diverse group of listeners. We have both superintendents, we have nonprofit leaders, EdTech leaders, we have government folks that come on this podcast and i-Ready, which is one of Curriculum Associates subsidiaries, i-Ready data from the last couple months where we're talking in September, 2024, you all released a bunch of data and some insights into that data very recently and it was profound. So why don't you talk us through the report that I'm referencing here, Confidence and Difference, what we know now about where students are.

Dr. Kelsey Young: Yeah, so we released this summer, a couple different reports, one in end of June looking at student growth post pandemic. So kind of plotting their academic trajectory across three years for five cohorts of students and then comparing that to a pre pandemic trajectory to see how student growth has kind of varied. And the other report, the one you're referencing is our annual State of Student Learning report, which looks at the end of this school year what percentage of students have reached grade level. And then we compare that to prior years and then a year prior to the pandemic to kind of get that historical trend. I think across both of those studies, the general story is kind of the same. We're seeing these really lasting impacts of the pandemic that students are still not where they were pre pandemic, whether we measure that through their academic growth or just who's reaching grade level.

Hayley Spira-Bauer: And this is alarming. So if I take my educator hat off for a second and I think about all of my friends that are parents and are concerned about where their students are, where they should be, you have this deep seated rooted, you just worry that your child or the children in your school are not where they should be. But if you put your educator hat on and you look at the national perspective, this is profound and alarming. We have to really take stock using this information to guide how we operate. Now this might not, you might not have a perfect answer for this, but I'm curious, what has the reaction been to this data? What are you seeing over at Curriculum Associates when folks are digesting it?

Dr. Kelsey Young: Yeah, I think that's a great question. I think to your point, it's obviously been a little alarming. I think it's also been disheartening for some, I think especially people out in the field, educators who are putting in such effort to try and make up for the lost instructional time from the pandemic. We've poured funds at this. There's been a lot of effort to remedy some of these gaps, I think. So there's a little bit of, it's a disheartening trend to see this many years removed from the pandemic. I think at the same time, and you're probably very familiar with this as well, coming from education, it's not too unexpected. A lot of educators are finding this data fairly validating to say, okay, this isn't a my school, my district, my state trend. This is a national trend, especially for our State of Student Learning report. We create a nationally representative sample to make sure we're mimicking the makeup of the nation to speak to and be able to generalize these trends to say, okay, this is what we're seeing nationwide. And so I think there's some validation there as well. And then I think for those of us who've been in education,we know education change is slow.When we put interventions in place, we try new policies, we try new practices. It takes a really long time to see the effects of those. And so I think it's not too shocking to see the little impact that we've seen so far. I think the other thing we saw in a couple of our studies, but especially that growth study that was a little surprising for people was the impact of the pandemic on very young students. So I think there's a lot of natural assumptions are older elementary or middle school students had their educational career obviously greatly disrupted with the pandemic, but for our younger students, especially those students who weren't in school yet during the pandemic, I think there was a lot of expectation that because they weren't in school, the effect of the pandemic would maybe be less on 'em. And what some of our data is showing is that they are actually further behind where we would expect them to be than older cohorts. And so I think that was a little bit of a shock to people to see that finding. Now I think for those who are in early childhood who are developmental psychologists, this isn't too surprising. We know those years are so critical for developing foundational skills, for learning how to be school ready for all of these other skills that are absolutely necessary to come in and learn those academic skills that we're hoping students can build on. But that was definitely a little bit of a shock for people. I think there's also the fact that there's just less data for these really young students. We're one of the few assessment providers who do test down to K to two. And so we have that data to say, okay, yeah, these students are actually more impacted than we originally thought. State testing doesn't start until third grade for the most part. And so this was just some newer data, but I think a little bit surprising for some people.

Hayley Spira-Bauer: Okay, there's a lot to impact there. First of all, as a parent, no, it's great. This is why I love talking to you because one, I can get so nerdy and you are just as excited about this nerdy stuff. So first of all, as a parent to an eight and a 5-year-old, I can confirm that I was super, like during the pandemic, I was like, oh, they'll be fine. And I see the impact. I taught kindergarten, first grade, second grade, I know where my kids should be. And even though they were with me doing learning at home, they weren't in a classroom setting for 6, 7, 8 hours a day. So yeah, I feel that. But I want to go back to something else. So obviously you talked about how change is slow, so how are we supposed to feel as educators that both change is slow and we put all these interventions in place, but now the money has run out for them and these interventions, specifically high dosage tutoring cannot persist, cannot continue in many districts that simply don't allocate their or previously didn't allocate their funds to permit for that type of support?

Dr. Kelsey Young: Yeah, I think great question. I think I obviously don't want to speak to how people should feel about this other than maybe frustrated, but I get that I completely understand that that would be really a challenging situation to be in. I think one, I was just reading an article the other day that was looking at Covid relief funds and applications for extensions. And then I think every application for extension that's come through has been approved. And so I think there is opportunity though it's not perfect to say, okay, yeah, we haven't spent all of our funds. We need to extend this, apply for those extensions, get longer time periods to be able to spend those funds to be able to address some of this issue you're speaking to. I think the other thing that will be coming out, not from us, but NAEP will be releasing new data too. And so I think anytime new data is coming out from us, other assessment providers or the National Assessment of Educational Progress, there's always an opportunity for policy to kind of reconvene, reevaluate what's needed. And so I think a lot of us in the field are hopeful that at that time point we will have opportunities to make the case for or reevaluate what's needed to better support students at this point.

Hayley Spira-Bauer: I had a really great conversation with a few superintendents yesterday regarding this is a crucial moment for reevaluation of what to continue and what to get rid of because there is funding that is drying up. So let's assume that for some they spent their funds, there may not be renewal. You have to make some difficult decisions and maybe you decide to keep something you didn't previously have in place before the pandemic, but eliminate other items that are not effective. So speaking of efficacy, I know your dissertation is on implementation science, so let's review for folks that maybe didn't listen to your first episode, how do we make sure interventions that we know are backed by science are effective in their implementation?

Dr. Kelsey Young: Yeah, such a great question and I think something we really tried to emphasize in both of our research releases this summer only because yes, globally things looked maybe not where we wanted them, students are still not performing where they were pre pandemic. But within each of those studies, there were very tiny pockets that I don't want to oversell, buttiny, tiny pockets where we saw some recovery.We saw either growth at an accelerated rate, meaning students are moving closer to where they were pre pandemic or small increases in the proportion of grade level students. We saw that mostly in older students, older grades where we know there's a lot of targeted intervention for students who are again, participating in state testing, who are receiving high dosage tutoring programs like that. We also saw it actually in our grade K students, but only in phonics. But we also know there's been a lot of policy around the science of reading. So there's a lot of explanations that made sense for some of the very tiny places we saw recovery. And so I think what we really wanted to focus on in that study or in our results from those studies, we say, okay, hey, what can we learn from these tiny pockets of recovery? And what we think is happening is that there's a really great alignment between the population we're trying to serve and the intervention we've put in place. And so when we do that and we match those things, well, we can have really great results. And so I think our encouragement was for districts to evaluate their data in a more nuanced lens to say, okay, who is this working for? And then under what conditions is this intervention working? There was another report that came out sometime around ours, I think from Harvard looking at all of the barriers districts ran into and just trying to get the dosage of tutoring programs or their supplemental instruction programs, scaling it to reach the amount of students needed. And so I think really evaluating those barriers to understand what's getting in the way of realizing the benefit of that evidence-based practice,

Hayley Spira-Bauer: Always the good stuff. Kelsey, you're always tying it back to both what we know about implementation science, but also what we can learn from the data sets before us. Can you talk a little bit about for me, please, when you disaggregate into subpopulations what this data showed, when we talk about our most vulnerable populations, how did they fare? Was it what we typically see? Where there patterns? What are we feeling about our disaggregated data sets?

Dr. Kelsey Young: Yeah, great question that I think is something we were also trying to emphasize in our approach to the data as well is that it's super important to disaggregate data. I think mostly because we're kind of seeing, you see these general trends when you take aggregate data and say, okay, here's what this looks like. But then when you start to parse it by subpopulation, we might actually see these varying trends where some students are moving towards where they were pre pandemic and some students are actually moving further away. Something that the field has often referred to as K-shaped recovery, that widening gap in student performance between students who are below grade level or on or above grade level. And turns outwhen you average two lines that go in different directions, you get kind of a general trend, but it's not really informative for either population.And so that's where we've been really particular about making sure we disaggregate data. So now that I'm off of my data soapbox, what we were seeing was that a lot of students on grade level we're doing okay, staying on track with historical trends, students below grade level. On the other hand, were sometimes struggling to keep pace with their own historical growth or moving further away. Now, this varied again by cohorts, so age of students during the pandemic and how far behind students were our students who were just at grade level below. So a lot of times those bubble students, maybe your students who are receiving tier two intervention services actually made some really, really great recovery. But again, we know a lot of post pandemic recovery efforts, targeted supplemental instruction, things that work really, really well for our students who are on the bubble but maybe isn't quite enough for our students who are even further below. So we're seeing that. I think we're seeing when we disaggregate by community income or demographic groups, we see kind of these continued inequities. We did see trends that kind of mirrored each other, and so if there was recovery happening in this say fourth grade cohort, that was happening across all demographic groups, which was great, but again, given preexisting inequities, this isn't enough to remedy those. And so we're seeing those continue in very few cases. Those are widening a little bit, and so it's something we absolutely need to attend to. But again, I think it's just the limited progress we're seeing generally is translating into limited progress in our ability to address inequities in educational performance or educational attainment.

Hayley Spira-Bauer: So helpful to have this level of understanding of the data and also what it means for schools. I really appreciate that one because where I sit today allows me to have a good sense for how schools are spending their time, resources, money over the past four years and as well as looking forward, but also because I'm a nerd and I'm so passionate about standing where our system is failing our students and for whom it's failing our students and where our system is supportive of students. I'm super interested in this topic and appreciate that you have the depth of understanding that you do to be able to speak so eloquently to this topic. Kelsey, what are you hopeful for looking forward? What are you thinking like, okay, you have this data in hand, here's what I would do next if I'm a school leader or even a government official. What would you do if it was Dr. Kelsey Young sitting on Capitol Hill?

Dr. Kelsey Young: That's a great question. I think a couple things. I think where I am finding hope is these tiny pockets of recovery, especially when I can make sense of these tiny pockets of recovery given other policy and intervention efforts, it makes sense to me. And so I think there is some hope that these are paying off, that creating or investing in policy that targets the science of reading that targets really a phonics based reading education has shown up hopefully in an increase in the percentage of students in kindergarten who are grade level in their phonics performance that our students who started a grade level below are making progress and are actually kind of exceeding their own historical growth trajectories knowing that we've had a lot of supplemental instruction and targeted these students quite heavily in the past few years. And so I think paying attention to those trends, trying to make sense of what's happening with those trends and then trying to apply that to other areas. And so I think from a broader perspective, it is again, identifying the barriers. I'm running into implementing a program with fidelity or as designed, working with your either high dosage tutoring partner, your assessment provider, your curriculum provider, and saying, Hey, I want to do this for these students, but here's where I'm running into issues. I need your support in adapting this or making it easier for me to implement. And so this is again, where it's not always on the education system, but this is where we need to have a cohesive effort to address these barriers and break it down for educators and administrators to get the most out of the practices that they're hoping to implement. I think as far as a policy level, it's always easy to say, we need more funding for this because I think it is something that, again, we've shown that there are small increases in students on grade level and students' recovery, and so I think more funding is always an easy answer, harder to implement, but makes sense or has a lot of opportunity to just create more options for districts and have districts have more resources at their disposal.

Hayley Spira-Bauer: I love how you get so practical because it could be easy to turn to this and say theoretical, but my typical last question is what advice would you give an educator starting their career today? And I think you perfectly answered that with that prior question. I love the thought about looking at your school's particular barriers and then addressing those barriers one at a time until there are no longer barriers to be able to serve all your students effectively. Kelsey, it is always a pleasure to see you. Thank you so much for sharing this insight and coming back on the podcast. I love having you here.