New Mexico NAEP: Stalled Out, With Some Progress

by Seth Saavedra │Thursday, April 11th, 2018


NAEPery Circus

This past Tuesday at midnight 2017 NAEP scores went public. Amidst the hullabaloo that accompanies this ritual is a predictable chain of events:

  1. Four hundred "What to Expect" pieces
  2. Accidental releases of embargoed information
  3. Late night Twitter stalking of #NAEPDay
  4. RELEASE
  5. All too many all too hot takes
  6. Ambitious overreaches and alarmist shrieks
  7. Even more ambitious counter "think" pieces
  8. RESULTS ACTUALLY READ
  9. Level-headed commentary
  10. "Welp, what does NAEP actually tell us?"
  11. Next shiny edu-thing

Before I jump into the fray at stage nine, let's cover the NAEP basics.


NAEP?

Often called "The Nation's Report Card", NAEP is the National Assessment of Educational Progress, the U.S.'s most comprehensive view of what students know in a wide range of topics. It is a low stakes, demographically representative, and minimally changing assessment that tracks academic progress over time. The National Center for Education Statistics, in the U.S. Department of Education, administers the project. For many, NAEP is our best barometer of how much students have learned, particularly in fourth and eighth grades.


Across the U.S.

The NAEP story of the past decade has been disappointing, particularly the past four years. Some have labeled this period "Education's Lost Decade". Beyond the dramatic rhetoric, the lack of movement in student learning is troubling.

 

SOURCE: National Center for Education Statistics

 

The speculation and analysis about why, after fifteen years of upward progress, we have stalled runs the gamut. Experts point to everything from diminishing returns on education investment to hangover effects of The Great Recession to Federal Department of Education overreach.

 
 

As researchers dig further into restricted-use data over the coming weeks we'll gain additional insights about this great NAEP flattening. Meanwhile, us advocates, practitioners, parents, policy wonks, and business leaders must remain focused on bending the trend line upward again, especially for students of color and those from low-income backgrounds.


NAEP in New Mexico

While our sunsets in The Land of Enchantment are dazzling, our NAEP results aren't, often coming in last or next to last in the nation. This time around, only Louisiana and Puerto Rico consistently come in behind us.

 

SOURCE: National Center for Education Statistics

 

It's important to note NAEP's very real limitations. There is a natural temptation to compare scores over time. However, because completely different cohorts of students get tested every two years, this is tricky. Rather, as Matthew Chingos and Kristin Blagg advise:

A better way to compare and talk about NAEP performance is to use adjusted NAEP scores that account for demographic differences across students in each state. These adjusted scores allow for students to be compared with their demographically similar peers using factors such as race, receipt of special education services, and status as an English language learner. These are factors we know can affect test results, yet they are not shown in NAEP scores. The interactive tool below brings those adjusted NAEP scores to life.
— Urban Institute

So, let's do that. When controls are applied for age, race/ethnicity, frequency of English spoken at home, special education status, free or reduced-price lunch eligibility, and English language learner status, New Mexico moves up slightly. Below, the yellow lines are raw, unadjusted scores while the blue lines take into account the factors above. 

 

New Mexico 4th Grade Math

 

New Mexico 4th Grade Reading

 
 

New Mexico 8th Grade Math

 

New Mexico 8th Grade Reading

 

As you can see, New Mexico bumps up a bit with controls applied - on average 12 places upward. In 8th grade reading, we move from dead last to 26th. And, in three out of four areas, New Mexico has moved upward since 2009, marking tangible progress.

As the New Mexico Public Education Department shared via email, "About 2,500 4th and 8th grade students from 150 elementary and 120 middle schools in New Mexico took the NAEP exam online, for the first time ever, in reading and math." That's one of the values of NAEP, that it takes representative samplings from each state.

Next, let's look at demographic differences in NAEP scores across the state:

 
 
 

SOURCE: National Center for Education Statistics

 

As with the national results from the last decade, the results haven't changed much here. Though there are a few notable exceptions:

  • Since 2009, Hispanic students have grown in all areas;
  • African American students did not reach a statistically significant sample in 2015 or 2017, leaving out an important student population;
  • American Indian students remained flat in two areas, lost ground in another, and made growth in the fourth; and
  • White students saw growth in reading and backslid slightly in math in 2017.

What Does It All Mean?

My main takeaway for our 2017 NAEP results is two-fold: our Hispanic students are on the rise and we have a lot of work ahead. For all the ill conceived critiques of PARCC, NAEP is the antidote. It's low stakes. It takes a representative sample. It's every other year. It covers every state - and can control for demographic factors. And yet, New Mexico still falls far behind the pack.

Yet, we've made progress over the past decade, particularly for Hispanic students, even though it's stalled out recently. We have hard questions to ask ourselves. Where should our future investments in education be made? It's clear that "business as usual" only works for adults in the system, not our students. Where are our current bright spots and how do we scale them up? In districts such as Gadsden and Farmington, we see meaningful progress. How do we encourage more smart innovation in a sector that badly needs it?

And lastly, as Morgan Polikoff writes, "We need more rigorous investigation of these results to understand whether they can really tell us anything about policy effects." In other words, what impact do “college- and career-readiness standards” have on student achievement? What about the effects from so-called "community schools"? Much more research is needed here to better understand the early gains we made as a state and to get us moving upward again.



[6/19] For Our Future: This Week's Education News & More

Friends & Colleagues - 

As news of Secretary Skandera stepping aside came across the newswire, the predictable polarity around education made its inevitable appearance. While I am both thankful for the Secretary's service and excited to see what she pursues next, I'd be remiss not to say a few more words about her legacy in New Mexico.

Education is a divisive topic. The stakes are high and nearly everyone of us has attended some sort of school, making us all "experts" - with an opinion. Additionally, as a state we dedicate nearly 60% of our roughly $6 billion annual budget on preK through college education. Despite what many say, we spend about as much as we can on public education in a state which continues to languish economically. Do we spend that money as wisely or effectively as we need to? No. But is there much more blood to squeeze from our high desert stone? Nope; at least not until we modernize our economy, revolutionize our schools and reclaim our independence from the federal government and oil and gas.

Many people I love and respect disagree with the policies and practices of the NMPED under Secretary Skandera. That's all and good - and indicative of a healthy, robust republic. Let's have those policy and philosophical conversations. However, the degree to which Secretary Skandera's personal character has been and continues to be attacked is troublesome. As many of us know first hand, the realm of public leadership is an unending and often thankless one. That comes with the territory so you'll get no sympathy from me.

Though, for those of us who have met the Secretary, it's immediately apparent that she cares deeply about New Mexico and our students. She has relentlessly pursued an agenda to drastically improve education opportunities for all of our babies, regardless of skin tone or zip code. Was there much to be desired with communication from NMPED to teachers and parents? Absolutely. Has she ruffled feathers along the way?  Of course. But that's often what we need in my beloved homeland.

So, as we think about our next Secretary and Governor let's work collectively, in spite of our disagreements on education, to ensure our next leaders are as focused and invested in our students as Hanna has been. My work will be to ensure that no matter the political party of our next state leader, New Mexicans will prioritize education policies and practices to ensure our students become the future community, civic and business leaders we need. Now, here's this week's roundup:

  • [LOCAL: SURVEY] Proving that we have a long way to go in reimagining what's possible from public education in New Mexico, The Leadership Conference Education Fund’s second annual New Education Majority Poll found that three-quarters of Latino parents believe “U.S. public schools are doing a good job preparing Latino students for success,” a 10% improvement over 2016. Optimism is a noble cause though we have a long way to go to equalize a public education system which chronically underserves Hispanic and low-income students.
     
  • [NATIONAL: NEWS] The future of education is a system based on what our students have demonstrably learned instead of how long they've sat in assigned seats. In a big step away from the "Industrial Education" model of the 19th and 20th centuries, students graduating from Maine high schools must show they have mastered specific skills to earn a high school diploma. Maine is the first state to pass such a law, though the idea of valuing skills over credits is increasingly popular around the country. This approach is highly applicable here in New Mexico where we rely more than most states on homegrown talent graduating high school with the skills needed to create and join local businesses to drive our economy.
     
  • [NATIONAL: NEWS] As I've repeated over and over, spending money on education is important but it isn't everything. According to the74million.org, "Schools in the U.S. spent $344.3 billion on classroom instruction in fiscal year 2015, accounting for 60 percent of day-to-day expenditures, a figure that includes spending on salaries for teachers and instructional aides, according to the Census Bureau." So, even as spending on students ticks up nationally, our kids aren’t proving to be doing much better academically. Fourth- and eighth-graders across the country did worse in mathematics in the 2015 administration of the National Assessment of Educational Progress (NAEP) than they did in 2013, the last time the test was given. In reading, eighth-graders again performed worse in 2015 than in 2013, while fourth-graders’ scores remained stagnant. New Mexico is roughly in the middle of the pack for per-pupil spending at $9,725, yet we are dead last for nearly every NAEP category. We see yet again that the "how" of education spending is just as important as the "how much".
Nation Per-Pupil Spending + NAEP Scores