A couple of weeks back I read an article in Reason by Emma Camp, More States Are Using Science-Backed Reading Instruction. It Shouldn't Have Taken This Long., touting “science of reading” instruction methods, which stress training in phonics—mapping letters to sounds—as opposed to “balanced literacy”, aka “whole language”, aka “three cueing”, which, Emma says, was discredited back in the 1970s by, well, “science”, but remains “incredibly popular”, especially among progressive types in those benighted, unscientific blue states—thanks to which, “millions of kids have been set up to fail” (quoting Emily Hanford, writing in apmreports).
Fortunately, says Emma, there’s now proof that science-based reading is the real deal. Mississippi introduced a SBR program back in 2013, and Alabama and Louisiana followed suit in 2019. The results, says Emma, speak for themselves.
Mississippi's test scores skyrocketed. According to the Associated Press, in 2013, the state was ranked 49th in fourth-grade reading. In 2022, it ranked 21st. Low-income students saw particularly great benefits. The state went from ranking as one of the worst states in the country for low-income fourth-graders in 2013 to second in the nation in 2022.
Alabama and Louisiana followed suit with similar legislation in 2019 and saw their own gains in performance. In 2019, Alabama ranked 49th among low-income fourth graders, and in 2022, it ranked 27th. Louisiana was ranked 42nd in 2019 and is now 11th. Both states, according to the A.P., actually saw modest gains in reading scores during the pandemic.
This all struck me as a bit rosy, so I followed Emma’s link to the A.P. story. As it turns out, I was right. The A.P. story, ‘Mississippi miracle’: Kids’ reading scores have soared in Deep South states, by Sharon Lurve, is based on data provided by an interactive graph Lurve says she designed using data from the National Assessment of Educational Progress. Unfortunately, Lurve doesn’t understand how NAEP data “works”—and she doesn’t understand a lot of other things as well, as I’ll make clear in the rest of this article. (I am unable to embed this graph, so if you want to see it you’ll have to look at Lurve’s article.)
Am I an expert on NAEP? Well, yeah, I am, pretty much. I used to write about NAEP under contract with the National Center for Education Statistics for 19 years, from 1996 to 2015. But Lurve’s first and biggest mistake doesn’t have to do with NAEP data at all. Her (inaccurate) headline reads “Kids’ reading scores have soared in Deep South states”, but the actual head for her graph is “Gulf South rose in ranks for low-income1 kids' reading”, purporting to show that 1) if you look at the changes in “rank” for the states over time according the reading scores of their low-income students, you will see that the Gulf South States are “rising” and therefore 2) their kids’ scores are, well, “soaring”. But that is absurd. You can’t tell if the scores of students are improving based on changes in their “ranking”—even if you could “rank” states using NAEP data, which you can’t.
Suppose the scores for students in all states fell from 2013 to 2022, but scores for students in Mississippi fell less than average. Then Mississippi’s rank would “rise”, even though students were learning less. There is no necessary relationship between changes in a state’s “rank” and its students’ performance, so constructing a graph based on state rank and attempting to make meaningful statements about student performance based on that graph is absurd. If you want to talk about student performance, well, look at student performance!
If you do look at student performance what do you see? You see one honest, impressive nugget. Reading scores for low-income fourth-graders in Mississippi were 201 (on a 0-500 point scale)2 in 2013, 215 in 2019, and 212 in 2022. Comparable scores for Louisiana were 203, 202, and 204; for Alabama, 207, 199, and 201. NAEP scores are based on samples, and therefore are estimates, and apparent differences in estimates have to be tested for statistical significance. The increase for Mississippi from 2013 to 2022 is significant, as is the decline for Alabama. For Louisiana, the 1-point increase was not statistically significant. For comparison purposes, the national average reading score in 2022 for low-income fourth-graders was 203, and the score for hippy-dippy California was 201—no different than either Louisiana or Alabama.
I can wonder if Lurve got the idea for basing her story on state “ranking” rather than actual student scores from publicists for the companies selling reading programs to the three states. As I’ve already explained the idea is absurd, and this initial absurdity is squared by the fact that NAEP doesn’t attempt to provide a state ranking—any attempt to do so would provide statistically meaningless results—and, on top of that, the rankings Lurve does provide are completely wrong.
For example, Lurve tells us that
Mississippi went from being ranked the second-worst state in 2013 for fourth-grade reading to 21st in 2022. Louisiana and Alabama, meanwhile, were among only three states to see modest gains in fourth-grade reading during the pandemic, which saw massive learning setbacks in most other states.
The thing is, the graph says that Mississippi ranked 2nd in 2022, not 21st. The ranking Lurve gives us in the text is “reasonable” (almost), because, according to the NAEP Data Explorer, the average for Mississippi’s low-income students in 2022 was significantly higher than the scores of 29 other jurisdictions (making comparisons between Mississippi and the others on a one-to-one basis, not an overall ranking), though one of those jurisdictions was the District of Columbia, not a state, something that Lurve might have noticed if she had studied the data more closely. The statement in the graph, that Mississippi ranked 2nd, is not based on tests of statistical significance but rather simply “eyeballing” the state scores, and “ranking” Mississippi as second because the only state with a higher score is Florida—though in fact the difference between the two is not statistically significant. Lurve simply does not understand how to interpret NAEP data. (All the other “rankings” Lurve supplies in both the text and the graph are similarly flawed.)
There are a lot more problems with that little paragraph, and with the graph. Lurve tells us that Alabama and Louisiana experienced “modest” gains from 2019 to 2022. Actually, the differences in scores for low-income students for the two states between 2019 and 2022 were not statistically significant, so there was no gain at all, modest or otherwise, and certainly no “soaring”. (This also is true for scores for all fourth graders, not just low-income.) In addition, the scores for Alabama in both 2019 and 2022 were both below the score for 2013 by a statistically significant margin. Lurve is trying to fit the “good” story for Mississippi “across the Gulf”, as it were, to include both Louisiana and Alabama, and she simply misreads the data to make it fit.
The other story Emma Camp links to, Emily Hanford’s At a Loss for Words: How a flawed idea is teaching millions of kids to be poor readers, for apmreports (published by American Public Media), is far more substantial, but it has its problems as well, notably the notion that teaching kids phonics is the key to reading: if you teach kids phonics, they will learn to read; if you don’t, they won’t.
In fact, as Hanford admits a bit awkwardly in the course of her article, the “smarter kids” (the upper half of the distribution, as we like to say in the biz), will very likely learn to read no matter how you teach them. Learning phonics—mapping sounds to letters—is an acquired skill, but a great many people can acquire it without conscious effort. The real concern, obviously, is the lower half of the distribution. Is conscious phonics instruction the “secret” for teaching all kids to read fluently?
Well, no, but it does help. Marilyn Adams, whose “classic” work, Beginning to Read, published back in 1992, covers the reading wars that raged back then, between the very groups discussed in Hanford’s thoroughly researched piece. In her book, Adams cited research concluding that, other factors held constant, an emphasis on phonics instruction would increase the reading performance of low-performing students by a small but statistically significant margin. A conscious or unconscious grasp of phonics is necessary but far from sufficient for learning to read fluently, something that Hanford, though she interviewed Adams and quotes her in her post, does not discuss.
The limited impact of phonics-heavy instruction means that the sweeping conclusions so often voiced by advocates of such curricula—that “millions of kids can’t read”—are wildly overstated. They can read, but not well, and many—many but not all—could read a little better if they had better instruction emphasizing phonics.
This is not nothing. I have repeatedly pointed out, particularly in a couple of very long-winded articles, viz., CRT v. Anti-CRT: Wait, Wait! You’re BOTH Right! Occasionally. and A Limo at Risk, Part I: A skeptical view of American education reform, 1983-2017, that the “Big Blues”—New York and California—are consistently outperformed by the “Big Reds”—Texas and Florida, according to earlier NAEP data. Looking at the 2022 data for reading scores for low-income fourth-graders shows something a little different: New York (202) and California (201) are lower than Florida (215) and Mississippi (212), but so is Texas (202). Clearly, New York, California, and Texas all have something to learn from Mississippi and Florida.
However, this is still not the full story. If we look at the reading scores for low-income students at grade 8 in the states discussed here, we see the following: National, 248; Alabama, 238; California, 248; Florida, 253; Louisiana, 249; Mississippi, 247; New York, 251; and Texas, 248. Alabama is below the national average and Florida is above it. The others, blue and red alike, are not significantly different. If the next two assessments show an increase in scores for Mississippi, then their new reading program will have made a substantial impact. But at present, the jury is still out.
In any event, what happened in Mississippi—and not in either Louisiana or Alabama—is not the “miracle”—the “skyrocketing” scores that conservative critics of the American educational establishment are always searching for. During the second Bush administration, which of course made the most concerted educational reform effort of any administration in history, a group of smart, dedicated reformers set up a “model” reading program incorporating a strong phonics component, intending to show the “liberals” how it was done. After years of effort, the program produced no measurable improvements and was simply forgotten. I am sorry that I can’t give the citation for the final report, so you’ll have to take my word for it. I have no doubt that the Bush people were highly competent, and the program was well funded. But it did not make a measurable difference. Despite studies showing the benefits of reading instruction that emphasizes phonics, attempts to implement such programs on a large scale, in a variety of environments, have not met with success.
The simple fact is, educational performance is never going to “skyrocket”. What liberal reformers in the past, and conservative reformers in the present, are always searching for—and never finding, because it isn’t there—is that miracle on-off switch that turns low-performing students into—into, well, the hyper-competitive over achievers who invariably comprise the critics of the American educational system on both the left and right, the folks who moan ever so loudly that American schools are “lousy”, that they are a “disaster”, etc. etc., etc., because American public schools don’t reflect the elitist, upper-middle-class values these critics so unconsciously embody and so loudly, and, well, so “ignorantly”, espouse. This is an argument that I have made at length in my two long posts linked to earlier, CRT v. Anti-CRT and A Limo at Risk, though both posts lead off with a lot of “history”, so if you want the details you’ll have to do a fair amount of reading. But you do like reading, don’t you?
Afterwords
Many critics are particularly frustrated by the fact that the U.S. educational system seems much less elitist than that of many other countries, which is true. Well, if the U.S. had been ruled by privileged, land-owning aristocracies who monopolized all social, cultural, economic, and political power to themselves for centuries if not millennia, we’d would be like everyone else. But we weren’t, and we aren’t.
At his substack blog Slow Boring, Matt Yglesias has been running a series of long posts on the ins and outs of the politics of educational reform, the fourth and most recent of which is here, and includes links to the previous three. To my mind, Yglesias gives a very accurate picture of the many political forces militating against educational reform, but isn’t willing to bite the last, skeptical bullet, which I chew on so aggressively in the two posts referred to above, to wit, that the student, rather than the teacher, or the curriculum, or the administration, or anything else, is the key variable, along with their parents, and both groups resist marching to the upper-middle-class drum that Yglesias, like so many others, so earnestly and innocently beats.
Among the middle class, parents want their kids to be “happy” rather than challenged in high school, while their kids want to be popular rather than academic overachievers, which is why academics so often take second or third place compared to athletics and a wide variety of other time-consuming “extra-curricular activities” (they’re probably not called that any more). In the big cities, many voters see the city government as the employer not of last resort but rather first, which means that the first purpose of the school system is to provide large numbers of “good jobs”, which pay well, are undemanding, and, above all, offer lifetime security, considerations that, of course, guarantee that mediocrity will be rewarded, innovation discouraged, and failure ignored. A great many low-income students see a middle-class education as irrelevant to their lives, because they do not have middle-class values. Charter schools, the great hope of the Bush administration, can have real success, but that success is limited by the fact that they only appeal to parents and students with middle-class values, and many parents and students do not share these values.
The outstanding problem in American education, of course, is the performance gap between black and white (and Asian) students. This is not a problem that will be solved by educational reform. Many black young people lack any confidence in the values of “white” society and also lack confidence in their ability to master its demands. Many young black people live in a culture where they are far likelier to win the approval of their peers by rejecting white values rather than conforming to them. Reformers, with all the good will in the world, cannot “reach” someone who does not want to be reached.
This is not cynicism, and this is not racism—there is, in my opinion, no reason to believe that there is a genetic component to the differences in average black and white academic performance. However, I also believe that significant improvement in black educational performance can only be achieved through changes in society rather than changes in educational policies and practices. The changes I suggest, in my post CRT v. Anti-CRT, are not at all “revolutionary”—quite the contrary, in fact—and seriously un-woke—but, since they tread on so many entrenched special interests, inevitably encounter strong resistance whenever they’re attempted.
1. To be precise, these are students coming from low-income families, as determined by the students’ eligibility for subsidized lunches under the National School Lunch Program. In NAEP terms, “NSLP-eligible”.
2. NAEP plots scores for fourth, eighth, and twelfth graders on the same scale, which is why it is so large.
