When the scores on the 2013 National Assessment of Educational Progress (NAEP) were released, much was made of gains registered in Washington, DC, which led the nation in rate of improvement. Schools Chancellor Kaya Henderson called them “breakthrough gains” and attributed the increases to a stronger curriculum, better teachers, and the District’s ‘get tough’ approach to evaluating teachers. She told the New York Times, “When you raise expectations for students and teachers, they rise to the challenge and produce.” The Times noted that the “get tough” approach preceded Henderson’s tenure. “The district’s new policies, initiated by the former chancellor, Michelle A. Rhee, have come under criticism from teachers’ unions and others who say they put too much emphasis on test scores,” Motoko Rich wrote.

Teach for America issued a press release praising TFA alumna Henderson.{{1}} All in all, these NAEP improvements represented, much of the press and many politicians said or implied, the triumph of the no-nonsense, “get tough on teachers” approach begun by Rhee.

I understand spin and the desire of those responsible for current policies to want to make things look good, but the rest of us need to take a deep breath and a second look.

In fact, a closer look at the DC data reveals all sorts of contradictions. It raises the possibility that DC is celebrating prematurely. It could be that reports of the triumph of ‘get tough’ policies are misleading–and perhaps just plain wrong.

You know the old saw about “Lies, Damn Lies and Statistics,” of course. I offer a variation: “Lies, Damn Lies, and Headline Writers.” Take your pick, because the DC NAEP press reports could have been headlined “DC Achievement Gap Grows Wider.”

Or “District Schools Tied with Mississippi for Worst in the Nation.”

Or “DC’s 20 Years of Educational Progress Continues at Same Rate.”

And so with that, let’s take another, deeper look. NAEP scores began rising in Washington long before Rhee arrived in the summer of 2007. Take 4th grade math, for example. Fourth graders scored 193 in 1992, and in 2013 scored 229, a dramatic rise of 36 points in 17 years. But they had jumped to 214 before she was hired, meaning that 21 points of that 36-point gain, 60% of it, did not occur on Rhee’s watch or result from her policies.

Or look at 8th grade math, which has improved from 231 in the early 1990s to 265 in 2013, a gain of 34 points over 21 years. Again, much of the credit ought to go to those running the schools before Rhee arrived, because 17 points–half the gain–occurred before she came to Washington.

Graph the changes, as Guy Brandenburg has done, and you see that the steep climb began long before Rhee.

That the improvement has continued is praiseworthy. However, what most reports do not mention is that DC is still bringing up the rear, roughly even with Mississippi.

And if you dig deeper into the data, a disturbing picture emerges. Michelle Rhee came to Washington determined to close ‘The Achievement Gap,’ but–as I have reported before –it widened on her watch. The performance gap between higher performing students (those at the 75th percentile) and lower performing students (those at the 25th percentile) is now 47 points, and that is SIX points larger than it was in 1992.{{2}} Rhee and Henderson have widened The Achievement Gap, but of course they are not issuing press releases about that.

But maybe it’s not on them. There’s likely a demographic explanation, because DC is gentrifying. The percentage of White students has increased from 4.7 in 2003 to 11.2 today. Given DC’s history of racial and economic inequity, it’s likely that most of those new White families are middle- or upper-income, and we know that test scores are highly correlated with family income. That–and not the Rhee/Henderson policies–could explain both the gap and the NAEP increases.

The percentage of DC students scoring at Basic or Above increased from a mere 24% in 1992 to 66% today. That’s a praiseworthy jump, but the percentage of students scoring BELOW Basic remains at 34%, as contrasted to the national average of just 18%.

Combine “Basic” and “Below Basic,” and the news is still not good. Nationally, 59% of students are in that combined category, but in Washington an astonishing 73% of students score at that level. ‘Basic’ amounts to a grade of C, hardly a cause for celebration; rather, it’s questionable whether students scoring at Basic or below are adequately prepared for a fast-changing world.

Celebrating is premature for two other reasons: First, we don’t know how much of the increase can be attributed to students in private schools and public charter schools, close to 40% of total student enrollment.{{3}} Second, DCPS muddies the waters. It treats the scores of higher income students as ‘low income’ if they happen to attend a school where 40% qualify for free or reduced price lunch. That’s explained in detail in the footnote, {{4}} but the bottom line is that scores identified as ‘low income’ cannot be relied upon. That change is “masking whatever is actually happening,” Jack Buckley, the Commissioner of the National Center for Education Statistics, told the Washington Post. He cautioned against relying on the 2013 results to draw conclusions about the progress of the District’s low income children.

Sadly, DC students score significantly below students in every state except Mississippi; the two are now in what amounts to a dead heat for last place in the academic race to the bottom. For more state-by-state comparisons, see Gary Rubenstein’s blog or the NAEP website.

About the reporting: A small handful of critics, including Diane Ravitch and Bruce Baker, pointed out the contradictions that most of the press and politicians like Secretary Arne Duncan overlooked: the ‘get tough’ approach did not work across the board. Scores went up in DC and Tennessee and to a lesser degree in Indiana (all of which use what Ravitch calls “Test and Punish” strategies), but NAEP scores were stagnant or even declined in the “get tough” states of Wisconsin, Colorado, Delaware, Louisiana, Rhode Island, Connecticut and North Carolina. As Ravitch asks in her blog, “If test-and-punish strategies work, why don’t they work everywhere?”

Rhee is trying to rebuild her reputation, which is in tatters after USA Today, Frontline and this blog revealed the extent of the cheating on her watch and, more importantly, her failure to investigate. Given that reality, the press needs to be more vigilant and skeptical. Of course, she and Henderson will boast about these results, but there’s far less than meets the eye. And there’s nothing that I can see to support a “get tough on teachers” approach as a way of improving educational opportunities for children.

Since leaving Washington, Michelle Rhee has lobbied aggressively for ‘get tough’ teacher evaluation policies, with some success. But most places that are doing as she encourages did not do particularly well on NAEP, as Bruce Baker skillfully notes in his blog, School Finance 101.

Perhaps by now state policymakers and politicians have figured out that a “buyer beware” approach is in order when it comes to the “get tough on teachers” policies that Michelle Rhee and her lobbying group are peddling.

—-

[[1]]1. But pointedly did not mention Rhee, although she is arguably TFA’s most prominent graduate.[[1]]

[[2]]2. Another report says the gap was 55 points in 2013 and 62 in 1992, which would mean that it’s now slightly smaller. Whatever the 21-year spread may be, the gap between the groups widened on Rhee’s watch.[[2]]

[[3]]3. That information should become available in mid-December when the analysis of large school districts is released.[[3]]

[[4]]4. This from Mary Levy, a widely respected analyst: “DC changed the basis on which students receive free lunch in 2013, as described in more detail in the attached.  Most schools have stopped collecting the income-level forms.  Instead of free lunches for students whose families had submitted forms stating that their income was below the cut-off, all students at schools where at least 40% of the students are homeless, in foster care, or are from families receiving TANF or food stamps receive free lunch.  Most of the schools, both DCPS and charter, are in this category and are reported as having 99% free-lunch students, a big change from prior years.  This resulted in increasing the reported percentage of DCPS low-income students to 77%, which matches the percentage of DC NAEP test-takers.  This means that students in many schools who are not in fact low income would have their NAEP scores reported as free-lunch eligible.  Since these students in the past had significantly higher NAEP scores on average, shifting their scores into the low-income category almost certainly raised the low-income average.”

From the Washington Post:  “It is nearly impossible to track the performance of poor children because the method for identifying low-income students in the District has changed since 2011.

A child’s poverty status is measured by their eligibility for a free or reduced-price lunch. Until last year, children became eligible for free meals by turning in forms showing household income. Now, if 40 percent of children in a D.C. school are in foster care, homeless or receive welfare benefits, every child in the school is deemed eligible for free meals.

The change in the District is a test of a new federal policy meant to ensure that more hungry kids have access to free meals. It means that some children who are not actually poor but who attend high-poverty schools are now included in the low-income category, said Jack Buckley, commissioner of the National Center for Education Statistics.

That change is “masking whatever is actually happening,” said Buckley, who said his office is concerned about and working to address the inability to track the progress of poor children. He cautioned against drawing conclusions about the progress of the District’s poor children based on the 2013 test results.”[[4]]