New Mexico NAEP: Stalled Out, With Some Progress

by Seth Saavedra │Thursday, April 11th, 2018


NAEPery Circus

This past Tuesday at midnight 2017 NAEP scores went public. Amidst the hullabaloo that accompanies this ritual is a predictable chain of events:

  1. Four hundred "What to Expect" pieces
  2. Accidental releases of embargoed information
  3. Late night Twitter stalking of #NAEPDay
  4. RELEASE
  5. All too many all too hot takes
  6. Ambitious overreaches and alarmist shrieks
  7. Even more ambitious counter "think" pieces
  8. RESULTS ACTUALLY READ
  9. Level-headed commentary
  10. "Welp, what does NAEP actually tell us?"
  11. Next shiny edu-thing

Before I jump into the fray at stage nine, let's cover the NAEP basics.


NAEP?

Often called "The Nation's Report Card", NAEP is the National Assessment of Educational Progress, the U.S.'s most comprehensive view of what students know in a wide range of topics. It is a low stakes, demographically representative, and minimally changing assessment that tracks academic progress over time. The National Center for Education Statistics, in the U.S. Department of Education, administers the project. For many, NAEP is our best barometer of how much students have learned, particularly in fourth and eighth grades.


Across the U.S.

The NAEP story of the past decade has been disappointing, particularly the past four years. Some have labeled this period "Education's Lost Decade". Beyond the dramatic rhetoric, the lack of movement in student learning is troubling.

 

SOURCE: National Center for Education Statistics

 

The speculation and analysis about why, after fifteen years of upward progress, we have stalled runs the gamut. Experts point to everything from diminishing returns on education investment to hangover effects of The Great Recession to Federal Department of Education overreach.

 
 

As researchers dig further into restricted-use data over the coming weeks we'll gain additional insights about this great NAEP flattening. Meanwhile, us advocates, practitioners, parents, policy wonks, and business leaders must remain focused on bending the trend line upward again, especially for students of color and those from low-income backgrounds.


NAEP in New Mexico

While our sunsets in The Land of Enchantment are dazzling, our NAEP results aren't, often coming in last or next to last in the nation. This time around, only Louisiana and Puerto Rico consistently come in behind us.

 

SOURCE: National Center for Education Statistics

 

It's important to note NAEP's very real limitations. There is a natural temptation to compare scores over time. However, because completely different cohorts of students get tested every two years, this is tricky. Rather, as Matthew Chingos and Kristin Blagg advise:

A better way to compare and talk about NAEP performance is to use adjusted NAEP scores that account for demographic differences across students in each state. These adjusted scores allow for students to be compared with their demographically similar peers using factors such as race, receipt of special education services, and status as an English language learner. These are factors we know can affect test results, yet they are not shown in NAEP scores. The interactive tool below brings those adjusted NAEP scores to life.
— Urban Institute

So, let's do that. When controls are applied for age, race/ethnicity, frequency of English spoken at home, special education status, free or reduced-price lunch eligibility, and English language learner status, New Mexico moves up slightly. Below, the yellow lines are raw, unadjusted scores while the blue lines take into account the factors above. 

 

New Mexico 4th Grade Math

 

New Mexico 4th Grade Reading

 
 

New Mexico 8th Grade Math

 

New Mexico 8th Grade Reading

 

As you can see, New Mexico bumps up a bit with controls applied - on average 12 places upward. In 8th grade reading, we move from dead last to 26th. And, in three out of four areas, New Mexico has moved upward since 2009, marking tangible progress.

As the New Mexico Public Education Department shared via email, "About 2,500 4th and 8th grade students from 150 elementary and 120 middle schools in New Mexico took the NAEP exam online, for the first time ever, in reading and math." That's one of the values of NAEP, that it takes representative samplings from each state.

Next, let's look at demographic differences in NAEP scores across the state:

 
 
 

SOURCE: National Center for Education Statistics

 

As with the national results from the last decade, the results haven't changed much here. Though there are a few notable exceptions:

  • Since 2009, Hispanic students have grown in all areas;
  • African American students did not reach a statistically significant sample in 2015 or 2017, leaving out an important student population;
  • American Indian students remained flat in two areas, lost ground in another, and made growth in the fourth; and
  • White students saw growth in reading and backslid slightly in math in 2017.

What Does It All Mean?

My main takeaway for our 2017 NAEP results is two-fold: our Hispanic students are on the rise and we have a lot of work ahead. For all the ill conceived critiques of PARCC, NAEP is the antidote. It's low stakes. It takes a representative sample. It's every other year. It covers every state - and can control for demographic factors. And yet, New Mexico still falls far behind the pack.

Yet, we've made progress over the past decade, particularly for Hispanic students, even though it's stalled out recently. We have hard questions to ask ourselves. Where should our future investments in education be made? It's clear that "business as usual" only works for adults in the system, not our students. Where are our current bright spots and how do we scale them up? In districts such as Gadsden and Farmington, we see meaningful progress. How do we encourage more smart innovation in a sector that badly needs it?

And lastly, as Morgan Polikoff writes, "We need more rigorous investigation of these results to understand whether they can really tell us anything about policy effects." In other words, what impact do “college- and career-readiness standards” have on student achievement? What about the effects from so-called "community schools"? Much more research is needed here to better understand the early gains we made as a state and to get us moving upward again.