Assessment and Progress Monitoring in Early Childhood

Assessment and Progress Monitoring in Early Childhood

Front Porch Series Broadcast Calls‌

Gail Joseph: Hello, and welcome to the Front Porch Series. I'm Gail Joseph, the co-director of the National Center on Quality Teaching and Learning. And on behalf of my colleagues here at NCQTL and the Office of Head Start, who funds us, it is my pleasure to welcome you to another hour – or 45 minutes, I should say, on the front porch. The Front Porch Series of broadcast conference calls provides us and Head Start, and in the field in general, an opportunity to gather as a community of practice and learn from national experts from around the country on topics related to high-quality teaching and learning for young children.

One topic that is of critical importance when we're thinking about the quality of teaching and learning is that of ongoing progress monitoring, or just really understanding how are our children doing towards these school readiness goals. And so because of that, we are so pleased to have joining us today Dr.

Scott McConnell from the University of Minnesota. Scott McConnell writes a lot on this topic and is actually the developer of a tool that he might mention. He is a professor in educational psychology and also the current director of community engagement for the Center for Early Education and Development. So, we are so pleased to have him join us to talk with us for about half an hour, and then we will be responding to your questions after that. So at this point, I'm going to hand it over to Scott.

Dr. Scott McConnell: Thanks, Gail; and hello, everybody. It's – I was going to say it's nice to see you. It's nice to be talking to you, I guess. As Gail mentioned, I'm going to spend about a half an hour talking about assessment and progress monitoring in early childhood. Gail also mentioned that this is a product of the National Center on Quality Teaching and Learning, but here's contact information for me. And as I'm going through these slides, if you'll remember that website address, www.umn.edu/~smcconne, I'll put a copy of these slides up there later on today or tomorrow and you'll be able to get them there.

What I'm hoping to do today is to talk a little bit about school readiness to kind of frame our conversation about progress monitoring around things that I think are particularly important for those of us that work with Head Start and other income-eligible kids in American schools. Just – so to think a little bit about school readiness, and then to talk just a bit about sort of why and how we go about the – the task of monitoring kids' development of school readiness. I want to talk just a bit about response to intervention, a concept that's probably known to many of you but perhaps new to some of you, and then talk about two different approaches that are available to us in early childhood for monitoring kids' progress on things like school readiness. And then as Gail mentioned, we'll hopefully have some time for discussion and questions after that.

So, school readiness – what is school readiness? We talk about it a lot. There are a variety of definitions, some written into law and into regulations that matter for us, others that are offered in the – the professional literature, both researchers and practitioners. But I think for – for our topic today, or for – for our discussion today, we can think of it as this: a set of competencies and skills that enable children to adjust to, participate in, and benefit from K-12 education. So, it's a set of competencies and skills. It's not a single thing, but rather it seems to be a variety of skills and – and capacities that kids bring to the task.

It's both characteristics and skills. The difference, I think, between characteristics are things that are a little more stable, a little more consistent across kids, a little harder to teach in isolation. Skills are the opposite. They're more specific. We can usually teach them fairly – fairly directly, and they're things that we oftentimes include in our – in our programs.

We also can think of these things as sort of the entry point skills for children that – that allow them – think of our kids as graduating into kindergarten, if you will, and – and that once they get there, they've got to – to make adjustments to the – the sort of new way of life: to being a part of a bigger group; to participating as one of many kids in a – in a classroom; to listening to a teacher that – that they may not have a lot of familiarity or experience with; and to be able to take advantage of all of the different things that present themselves in kindergarten and the grades that come afterwards.

So school readiness is this sort of bundle of things that kids are and – and things that kids do and that we can influence and affect, but there's also a part that – that I think is really important for us to mention, particularly in presentations like this where I assume that we've got people from all over the country, and that is that school readiness also matters. It also matters where you live, that there are setting- specific skills and knowledge, or what Lizbeth Vincent some years ago called "the criterion of the next environment," that matter.

So there are a lot of skills about going to kindergarten, but there are skills about going to that kindergarten right down the street that matter differently. What does that kindergarten teacher expect? What kind of skills and procedures are important in that particular kindergarten? Those too have to affect how we think about, teach, and assess school readiness in – in our own individual preschool classrooms.

So when we think about school readiness, a couple of things. There are a variety of things that we know are important in school readiness. One is language development. More than anything else, if – if I were – if I were stuck on a desert island and I could only have one thing to help preschool kids with, it would be language development. It seems to be central to so much of what kids do, how they grow up, how they engage the world around them. So talking, listening, knowing lots of words, knowing what those words mean, being able to use those words in interesting and important ways, really, really important for young kids' development and for – particularly for their acquisition of school readiness.

Second, pretty important – couple – are phonological awareness development and child and teacher interaction skills. So, phonological awareness development is – is something that, for those of us in preschool programs, has become sort of an – an increasingly important part of our conversation. It's basically kids being able to use words as tools – to be able to hear rhymes, to hear parts of words, to break words up into their component parts, be able to put those component parts back together into one thing – so that they can sort of treat words as objects, which is very, very important when they start to read, when they start to sound out words and have to kind of collapse all of those funny sounds together into something that they know and understand.

But there are also child and teacher social interaction skills. These are the skills – and there are different sets of skills for getting along with kids and for getting along with teachers, but that allow an individual child to make friends, to engage teachers in socially appropriate ways, and to get both other kids and teachers to seek those children out and give them new and interesting experiences. That's pretty closely related to "kindergarten survival skills."

This is another Lizbeth Vincent term from probably 30 years ago. And – and I think what Dr. Vincent meant by it was the things that kids – sort of the soft skills that children need to be able to benefit from large group instruction in kindergarten programs. So it's things like knowing how to manage your materials; knowing how to hang up your coat in your cubby; for those of us that live in the north, how to get your snow boots off and hang up – and hang up your snow coat in the – in the cubby. It's the things that help children basically kind of manage the everyday activities and responsibilities of being a kindergartner.

And then the last part is just other content skills that are things that are specific to a kindergarten in a particular environment. So if you're here in Minnesota, it probably helps to know how to play outside in the snow. If you're at a beach community, it probably helps to understand about beach safety. So, it's the things that are specific to the places where kids live.

So, we – I've talked a little bit about what we want kids to do. We want them to be school-ready when they – when they come to the kindergarten door as a five-year-old. So how do we go about the task of monitoring their development? Well, let's – let's see if we can unpack that term. We'll start with development. My dictionary on my Microsoft Word told me that development is basically the process of changing and becoming larger, stronger, or more impressive, successful, or advanced. So it's clearly this notion of sort of growing up, in the simplest sense of the word. Monitoring is the thing that we do when we check on something at regular intervals, where we – where we find out if it's progressing or developing. So you put those two things together, and monitoring development might suggest that we're watching regularly, perhaps frequently, to notice how kids are developing this thing that we're calling school readiness.

So as we approach this challenge of monitoring development for school readiness, we can ask ourselves a couple of questions. First, why would we do it? Why would we care about monitoring school readiness for young kids? Well, probably the best reason is to identify individual students who might need different intervention, or to make sure that any different intervention that we offer to individual kids or to groups of children are actually achieving the goals that we've set out to achieve. We're clearly aware of the fact that not one size fits all. And so, by monitoring young kids' development, we can actually figure out which kids the one size we have been offering isn't fitting and – and figure out how to give them something different.

In addition, we might think about when we monitor school – school readiness. What I'll be suggesting as we go through this – this session today is that we will – that we should monitor school readiness development for all kids periodically. So for four-year-old programs, a lot of programs do it in the fall and winter and spring. That's – that seems perfectly reasonable, but it's for all of the children in fall and winter and spring so that we can ensure that nobody is falling through the cracks or nobody's missing our attention when they might benefit from something a little bit different, a little bit more.

As we do those periodic assessments, though, we're going to identify kids that might benefit from that additional bit of intervention. Those children we'll probably want to monitor more frequently. Those are kids that we're trying something a little bit different, and we want to make sure that that little bit different thing that we're trying is actually working. So staying in touch with their programs, with their development, with their growth over time is very, very important, particularly so that we can adjust interventions when they're needed. And how do we do this? The – probably the dominant approach to this, particularly in the K-12 world right now, but increasingly in early childhood, is this notion of response to intervention.

This is a well-circulated triangle of response to intervention, and – and this particular one talks about both kids' academic or pre-academic development and their behavioral development. But I think the – the part about this image that's most important is the notion that all of the children that we serve are in this triangle and that we assume – assume, and that's a really important assumption here – we assume that about 80 percent of the children will benefit. They will, in our case, acquire school readiness by our providing them high-quality, sort of general programs, the programs that we've all worked hard to design and that we think of as basically the classroom services.

Within the kids that we serve, though, there are about 15 percent of the children that, given that high- quality sort of Tier 1 or core of intervention services, this 15 percent of the kids, they won't quite make the progress that we wish they would. They'll be a little bit slower to acquire some skills or they might have a little bit harder time acquiring skills in sort of the general classroom program that we've developed. So, these kids we provide a little bit of supplemental service for, a supplemental intervention. A lot of these interventions look like smaller groups or more explicit instruction around particular content areas or additional time to – to acquire specific skills or – or competencies that are important in a classroom setting.

Another five percent of the kids are kids that we think might be eligible for what we call – what we typically call Tier 3 services. And these Tier 3 services are much more intensive, they're much more individualized, and they're, again, intended to help kids acquire the general skills and competencies, for instance, of school readiness, but to do so in a much more focused and much more intensive way. So this triangle that – basically suggests that there are kids in sort of three tiers or groups, and that the – that the Tier 1 kids are really, really – it's a big group of kids, 80 percent, and the Tier 3 kids, that's a small group of kids, just five percent, but that all of these kids are part of our overall responsibility and our overall attention to the development of school readiness.

So what does it look like to screen or to – to monitor development for kids in this RTI model? Well, the first thing that we would want to do is to assess behaviors of these individual kids that are related to the outcomes that we want for them; that we keep a tight focus on the things that we're assessing and the things that we're teaching.

Secondly, we'd want our assessments to be efficient and economical. Remember I suggested that we might do this for all of the kids in a classroom, it might be 20 or 25 kids, and that we're going to do that three or maybe even four times for all of those children every year. So, anything that's fast and that's cheaper to use is going to be better. Things that are longer and that are more expensive are going to be harder for us to do over time. We want – at least according to – to my thinking, we want assessments that are standardized and replicable. That is, that when you do it and when I do it, we get similar results. It's not our – it's not my judgment or my perspective versus yours, it's how the child's doing. And so, to the extent that the procedures guide me and guide you in applying the assessments similarly, that seems to be an advantage.

We also want these measures to be repeatable over time. So when I do a fall assessment and then a winter assessment, I can compare the two and I can ask myself the question, "How is the child doing? Is she or he actually growing or developing over time?" That means that we want these measures to be technically adequate. This is – that's sort of the traditional test for tests and measures, that they have reliability and validity, and that that's been built into the tests from the outset.

And then, finally, and perhaps most importantly, we want these measures to be sensitive to children's growth over time and to the effects of intervention. So as children change, we want the assessments to be able to describe the changes that they're making. Broadly, there are two approaches to this kind of child assessment or monitoring of child development. They're what I'm going to call developmental skill mastery monitoring approaches and what I'll call general outcomes measurement approaches. You are familiar, I bet, with at least one if not both of these.

The developmental skills mastery monitoring approach I think is one that's very well – well understood and fairly common in early childhood, so let me talk about it very briefly. Basically, a developmental skills mastery monitoring approach is one where we have a series of behaviors or items, and they're typically ordered developmentally. So we have three-year-old items or we might have 36- to 39-month- old items and then 40- to 43-month-old items and 46- to 52-month-old items. And the assumption is that kids are moving down that list of items, that they're acquiring skills that help them do better and better and better further and further and further down the list.

What this means is that our assessment relies on our ability to describe the child's performance of that particular skill, that it was naming three colors and now it's naming six shapes, and – so that – so that the performance that we notice is that the child is acquiring the skills that are sort of appropriate for their age.

When we think about these measures, what we want first is to make sure that the items that are on it are absolutely consistent with what we're teaching kids. We never – it's – it's kind of a drag to have an assessment tool that measures something different than what we're teaching, because we might be doing a great job at teaching and the assessment tool won't suggest that that's the case. So, making sure that these – these items that are included in a developmental skill mastery monitoring approach match up well with our own goals and – and instructional objectives is – is critical.

Secondly, we want these measures to describe short-term developmental accomplishments, so the kinds of changes that kids make over very, very short periods of time. We might do – in fact, in – in some special education instances, we do this kind of mastery monitoring every day. So – so, we want these measures to be highly sensitive to – to the skills that kids are acquiring. These measures tend to be very focused, so they – they talk about specific skills that children have and not sort of general characteristics that the children acquire, and they shift over time.

Remember I mentioned that we might go from naming colors to naming shapes, and this – this actually is, at least from my perspective, one of the challenges in these more skill-based approaches in that it's hard for us to be able to describe a rate of change or a rate of growth for individual kids. We don't know if they're moving along at a rate that helps us be sure that they get to full school readiness by the time they go to kindergarten. So, that's kind of a blast through of developmental skills mastery monitoring.

Let me contrast that with general outcomes measurement. So general outcomes measurement is a little bit different in that the goal – and – and frankly, the thing that we assess – doesn't change; the kid's performance changes. So, we have a consistent goal across time. It might be, for instance, we have an oral language goal that's – that the child can express – express their needs and wants and communicate with others. So that's the goal that we're assessing, and we assess it repeatedly by having the same task and seeing the child do better and better and better on the same task over time. On the right-hand side of the screen, you see a height and weight chart, a figure that I bet many of us are familiar with, where the way that we notice change is that a child is growing or a child is getting heavier. We're not changing the assessment; we're doing the exactly the same assessment. It's the child's performance over time that's – that's mattering.

So these – what I want to talk about are some examples of general outcome measures that we call individual growth and development indicators, or IGDIS. These IGDIS, as it says, focus on individual children. So it's about how little Scotty is doing or how little Gail is doing, not about how all kids are doing. But we can aggregate them across kids so that we know how groups of children are doing. These measures describe growth and development. That is, they show how a child is doing today and how she or he has done over the last period of time. And they're – they're indicators. They're not direct assessments of the – of the things that we care most about, but they tell us how kids are doing, just like height and weight tell us about how a child's overall health and well-being is going.

There are two sets of these IGDIS right now, a set that we – that we first developed that we call IGDIS

1.0. There are three of them. The first one is a picture-naming task. This is a measure of oral language.

We show kids pictures of things, like these bananas, and we say, "What is that?" and we count how many of those everyday objects they can name in a minute. That one-minute score is a great overall indicator of their general language capacity, both their vocabulary and their overall spoken language.

We have two measures of phonological awareness. This is one of alliteration. So, we show this card to the child and we say, "Listen to me. That's rain, house, rake, pig. Which one on the bottom starts with the same sound as rain?" and we teach the kids this task. And so, our – our objective of course is for them to point to the rake because it also starts with an "r" sound. Then we have a similar measure that's a rhyming measure. "This is bees, pants, gate, cheese. Which one sounds the same as bees?" So again, we're interested in kids identifying the distracter at the bottom that matches, here rhymes, with the target stimulus at the top: cheese, bees. Depends on which one you were looking at.

These are measures that we – we started developing about 15 years ago. And what we know for these measures is that they're very easy to administer. We've implemented these measures in a good number of Head Start and early childhood ed programs across the country. Teachers, assistant teachers find them relatively easy to use and to use with fidelity. The scores that they produce are pretty understandable to teachers and parents. We like that, that parents and teachers can look at a score and kind of get the same information out of it. They tell us about differences between younger and older kids and kids with and without disabilities.

And perhaps most importantly, they're both correlated with criterion measures, other, longer measures of – of kids' school readiness development, and they show growth over time. So, we can draw – draw charts that are a lot like that height and weight chart that allow us to be able to monitor whether kids are making adequate progress given the time that they have left to get to school readiness goals before they enter kindergarten.

But the research that we did over the last 15 years also suggested that these measures, the ones that I've shown you here, are not particularly sensitive, particularly for lower performing kids. So kids who have lower language skills or who are just learning how to do rhyming and alliteration, these measures are not as good for them. They tend to – to not score in a range where we can describe their performance.

So, we set out to design a new set of IGDIS that we cleverly called IGDIS 2.0. And here our – our goal was to hold on to the principles of general outcome measurement but to use a more contemporary approach to measurement design, an approach called item response theory, to sort of underlie what these measures do. So far we've developed a set of – of scales across four different domains.

We have actually five measures: so we have an oral language measure, two phonological awareness measures, an alphabet knowledge measure, and a listening comprehension measure. And we've got lots and lots and lots of items that allow us to measure those – those skills, primarily for four- and older three-year-old kids, but particularly kids who are just learning English and just – just acquiring sort of sophisticated language.

We've taken all of those items that we've developed and – and organized them into what we call seasonal sets. So there's a fall set of items, a winter set of items, and a spring set of items that allow us to really isolate whether kids are meeting sort of minimum expectations for development in each of those four domains. And then, we've identified ways to distinguish between kids that are doing well enough and kids that might need additional intervention in oral language or phonological awareness or alphabet knowledge or comprehension. We call those cut scores. If you score above the cut, you're probably doing okay. If you're scoring below the cut, we need to gather some more information, but you might need additional intervention.

We've also developed some evidence of the convergence of these measures – that is, that they correlate with – like the oral language measure correlates with other longer measures of oral language, and all of them correlate with kids that are making good progress towards demonstrated school readiness when they enter kindergarten. We're starting to see, in some of the work that we're doing as part of the Center for Response to Intervention in Early Childhood – and I'll come back to that in a second – we're starting to see a convergence between these measures and interventions. That is, that when we identify kids who might benefit from additional intervention and we provide the intervention, we see both that they do better in the intervention and they do better on the measures. And we're seeing that classrooms that are organized in ways that we think will promote kids' development actually are related to these measures. So, kids who are in good classrooms do better on these measures.

So we've been building other items and – and adding to the measures and constructs. We're actually just starting some work to do exactly this kind of assessment with kids who speak Spanish at home and who are primarily Spanish-speaking children. That's been a major issue in assessment of school readiness in the past. And last, we've developed sort of decision rules that we think will work in real- world classrooms so that we can identify subsets of kids in your classroom that might benefit from additional intervention and from additional services and supports to move along towards school readiness. So in a – in a really fast way, that's sort of an overview of where we are with this approach to monitoring kids' school readiness.

So in thinking more generally about this topic, about progress monitoring and school readiness for kids in Head Start and Head Start-like programs, how might we put it all together? Well, first, I would encourage you and your colleagues to think about what your goals are for the kids that are enrolled in your program. Some of those will be defined, for instance, by the Head Start grantees that are – that are in the audience today. Some of those will be defined by the Performance Standards. Some of those will be defined by your individual states and how they have defined kindergarten entry competencies or kindergarten – kindergarten early learning standards and goals.

So being clear about what's important in your community is the first part, but then quite often there are more program-specific variations of that. What does our program want to achieve for young kids, and – and particularly for the kids that we serve? And as you think about that standard of school readiness, think about what you need to do to help kids achieve it. Given the children that you serve – the children and families that you serve, what are the essential features of a program that are going to help them meet those probably fairly aspirational or – or ambitious goals?

Secondly, what interventions and supports can you provide to reach these goals in two different sort of categories or buckets? First, what can you do universally? So children who enroll in your program, who show up in your classrooms, they're going to get the – the Tier 1, the general, the – the designed-for- everybody intervention. It's probably a pretty good intervention, but you want to think particularly about how to build elements into that, into that experience, into that universal program that allow kids to make the progress towards school readiness – that 80 percent or more of the children that we hope are in the Tier 1 services.

But we also know that – that there will be kids in those programs that in spite of our efforts to design a high-quality program, it won't be sufficient for them and we need to provide something a little bit different, maybe a little bit more intensive for them. So as you think about the design of your program to promote and to help kids develop school readiness, what are the tiers of intervention that you can add? How can you supplement the existing classroom activities in ways that will help children make the progress that you think are important?

With those two ideas in mind – first, what your goals are, and secondly, how you're going to go about achieving them – then I think you can ask yourself sort of the progress monitoring question. How can you identify all children's progress towards achieving those goals? How can you monitor for all of the kids that you serve the adequacy of the designs that – that you've constructed to make sure that they're getting what you want them to get? And within that, how might you adjust intervention and evaluate the – the effects of that adjustment for individual children that you serve? If it's a periodic assessment – fall, winter, and spring – where you identify some segment of the kids, how will you do that? What will be your criteria for selecting kids that might benefit from a little tiny bit more? That's going to be an essential part of building out sort of an assessment and progress monitoring program that will fit best in your – in your particular program.

Close