If you are looking for convincing evidence that “test-based accountability” and test-score obsessions are counter-productive, the ‘Canary in the Mine’ is the Broad Prize for Excellence in Urban Education. Without much publicity, the Broad Foundation did not award the $1,000,000 Broad Prize for Excellence in Urban Education in 2015 or 2016 and has no plans to begin awarding it again in the future.
Here’s why: It turns out that the NAEP scores of most of the Broad Prize winners (public school districts) have been flat for years. These districts have been living and dying by test scores, and it’s not working well enough to impress the Foundation’s judges.
Ben Weider of the blog 538 deconstructed the issue in a well-reasoned piece, “The Most Important Award in Public Education Struggles to Find Winners.” Not long after, the Foundation decided to ‘pause’ the $1 million award, citing ‘sluggish’ changes in urban schools. As Howard Blume of the Los Angeles Times has reported, billionaire philanthropist Eli Broad has shifted his focus to charter schools.
But that’s not really new news, as the Foundation’s own pie chart reveals. Since 1999, the Foundation has made $589,500,000 in education-related grants, and 24 percent of the money, $144,000,000, has gone directly to public charter schools. No doubt some of the ‘leadership’ and ‘governance’ dollars have gone to public charter schools, which make up 5 percent of all schools. Over that same time period, 3 percent of the money, $16,000,000, went to winners of the Urban Education Broad Prize ( mostly for college scholarships).
Mr. Broad hoped that urban districts could improve “if given the right models or if political roadblocks” (such as those he believes are presented by teachers unions) “could be overcome,” said Jeffrey Henig, professor of political science and education at Teachers College, Columbia University. The suspension of the prize for urban education could signal a “highly public step” toward the view that traditional districts “are incapable of reform,” Henig said. Mr. Broad seems to have already taken that step in his home city of Los Angeles, where he has been backing a concerted and expensive effort to greatly expand the charter sector.
Apparently it’s pretty simple for the folks administering the Broad Prize in Urban Education: Successful School Reform boils down to higher test scores. There is no public sign that anyone at the Foundation is questioning whether living and dying by test scores is a sensible pedagogy that benefits students. There is no public evidence that anyone at the Foundation has considered what might happen if poor urban students were exposed to a rich curriculum and veteran teachers, which is essentially the birthright of students in wealthy districts. Just the dismal conclusion that traditional districts are incapable of reform, followed by its decision to double down on charter management organizations, despite the truly offensive record of some of them of excluding special needs children and driving away students who seem likely to do poorly on standardized tests.
How sad…..
(This is excerpted from my forthcoming book, “Addicted to Reform: A 12-Step Program to Rescue Public Education,” which will be published by The New Press on August 1. It will be available in hardcover and as an e-book (the latter includes many videos from my long career).
(During my time at the PBS NewsHour, my non-profit production company, Learning Matters, received several grants from the Eli and Edythe Broad Foundation to enable us to cover Michelle Rhee in Washington, DC, and Paul Vallas in New Orleans. At no point did anyone from the Foundation ever attempt to influence our reporting, and I have the highest respect for our program officers there.)
The Merrow Report 
John, the essay below explains why and how schools (district & charter) should be assessed with multiple measures, instead of relying primarily on standardized tests.
This is a response to a recent report published by Fordham Institute which judges schools entirely on how well their students performed on standardized tests. Thanks for Michael Petrilli of Fordham, who was willing to publish this response on their website. My essay includes several specific examples, as well as resources for those interested in strong performance measures.
Reactions welcome.
Joe
http://bit.ly/2psPZvZ
Family and employer values should guide charter school evaluations
Joe Nathan, Ph.D.
May 09, 2017
While well intentioned, Fordham’s new report, Three Signs that a Proposed Charter School is at Risk of Failing, risks being a step backward for the charter movement. The study’s design misses several important aspects of the public’s attitude toward schooling, predictors of adult success, and advances in tools to assess a school’s impact. Policymakers and authorizers should be asking at least four key questions as they assess currently operating, and proposed new chartered public schools. Here’s a brief overview, and then the question.
Overview
From the beginning, Minnesota and many charter laws have included as one of their purposes to, as Minnesota puts it: “measure learning outcomes and create different and innovative forms of measuring outcomes.”
However, the report judges new chartered public schools in four states solely on “school-level student growth and academic proficiency data” on standardized tests. Researchers found that “student-centered” schools, such as those using a Waldorf or Montessori model, tend to struggle on these measures in their early years. Other forms of impact on students were not included.
The report prefaces the analysis with an acknowledgement that child-centered schools “aren’t ‘failing’ in the eyes of … parents who choose them [and] may not care if they have low ‘value added’ on test scores.” And it cautions that any use of its results “to automatically reject or fast-track an application” is a “misuse.”
That’s critical. Ignoring that caution would severely limit the quality and range of the schools that are opened and permitted to continue. Serving students well means using gathering and sharing a variety of information about schools and students, not just how well they score on standardized tests.
Perhaps, in future reports, Fordham can describe outstanding chartered and other public schools that use multiple measures. Multiple measures can provide a more comprehensive view of what schools are helping young people accomplish.
Four key questions for policymakers and authorizers to consider
First, and most importantly, shouldn’t they recognize that the public wisely cares about much more than a school’s “value added” on standardized tests? The 2016 PDK Education Survey, for example, found that more than 75 percent of the public rated several things as “extremely/very important for schools,” including “developing good work habits, providing factual information, enhancing critical thinking, preparing (students) to be good citizens, and preparing students to work well in groups.”
These are also the skills that America’s employers are seeking in recent college graduates, according to a 2016 study. More than 80 percent said they look for “evidence of leadership skills on the candidate’s resume, and nearly as many seek out indications that the candidate is able to work in a team. Employers also cited written communication skills, problem-solving skills, verbal communication skills, and a strong work ethic as important candidate attributes.”
Second, what are the best measures to predict success as an adult? One of the most powerful studies I’ve read in my forty-five-year career was published by ACT. Their researchers asked which of four factors best predicted success as an adult: high grades in high school, high grades in college, high scores on their test, and participation in debate, speech, drama, and student government.
ACT found that participation in those extracurriculars best predicted success in adulthood, as they defined it. Part of a school’s evaluation should therefore be the percentage of students involved in these programs.
Third, given that schools in a democratic society are not just places to prepare students for work, don’t we want young people to graduate schools with the tools and attitudes needed to be active citizens? A recent Education Commission of the States report, Mapping Opportunities for civic education, describes many benefits for preparing students, “not just for college and career, but also for citizenship and full participation in democratic life.” ECS documents that well developed engagement and service learning programs lowered dropout rates, reduced achievement gaps, improved school climate, and strengthened relationships between schools, students, parents, families, civic organizations, and community partners.
Finally, fourth, policymakers, educators, and authorizers should ask: “Do strong assessments exist beyond standardized tests that could help assess what’s happening with students in a school?” Fortunately, the answer is “yes.” For example:
The Stanford Center for Opportunity Policy in Education has created a curated collection of high quality performance assessments.
David Osborne produced Creating Measurement and Accountability Systems for 21st Century Schools: A Guide for State Policymakers.
The “Hope Survey” measures, among other things, whether students feel they can set and reach important goals. This instrument has been found to a powerful predictor of success in college.
Innovative district & charter educators have combined on efforts to promote a broader array of assessments.
The Center for School Change, where I work, has described many applied measures, including Alverno College’s public speaking assessment inventory and the New York State Performance assessments.
Fordham wisely urges authorizers not to reject certain educational models outright. Still, by focusing on test scores alone, the analysis sends the message that it is acceptable to continue relying on standardized tests as the central way of measuring a school’s quality.
Strong reading, writing, and math skills are vital. But Americans wisely want more from their schools. Students, the charter movement, and the broader society will gain if we:
Recognize the importance of assessing a broad array of skills and knowledge, not just those that are measured by standardized tests.
Refine and encourage use by states and authorizers of valid assessments that measure a broader array of skills and knowledge.
Support and encourage development of schools, chartered and otherwise, that help students develop many strong skills and broad knowledge.
Chartering has grown in part because it builds on the fundamental American values of choice within some limits, and the belief that those creating new products and services should expect to be judged on results. We wisely don’t judge cell phones, computers, other technological innovations, or fundamental freedoms like free speech on just one measure.
Relying primarily on standardized tests to evaluate chartered public schools is the wrong direction for this movement.
Opportunities to innovate and discover helped make the US a world leader. That energy should be a central part of chartering.
Joe Nathan, Ph.D., helped write the nation’s first charter law and has worked with the National Governors’ Association, governors, and legislators in more than thirty states to help develop, refine, and strengthen their charter laws. Professional, parent, and student groups have given him awards for his work as an urban public school teacher, administrator, researcher, and writer. He directs the Center for School Change. Email him at joe@centerforschoolchange.org.
LikeLike
John,
As someone who was a member of the Broad committee for about 12 years that reviewed the data and made recommendations to the Broad Jury, I can testify that there were many, many factors that were considered. We looked at gaps in graduation rates, attendance, finance and a host of other issues.I’m in Japan at the moment or I could give you a long list of the data points that we reviewed. In addition, all of the five districts that were finalists were visited by teams that looked “under the hood” at a host of other issues. Very important for me was looking at the trend lines over time.
You know Gregory McGinity at Broad. In fairness you should ask him and in the past they actually posted that data and gave each of the finalists that set of data.
I’m back home next Thursday.
Chris
On Sat, May 13, 2017 at 4:13 AM, The Merrow Report wrote:
> John Merrow posted: “If you are looking for convincing evidence that > “test-based accountability” and test-score obsessions are > counter-productive, the ‘Canary in the Mine’ is the Broad Prize for > Excellence in Urban Education. Without much publicity, the Broad > Foundation did ” >
LikeLike
Chris, While I do not doubt that your group looked at other data, I remain convinced that test scores were the central factor by far in the Foundation’s decision to suspend the Urban Prize. I base this on conversations I had–or overheard–at Broad events, including the last two award ceremonies. I was seated at Eli and Edythe’s table at the 2012 event at MOMA and have a strong memory of his frustration even then. At the 2014 event, which I also attended, I heard several conversations and participated in others about the disappointing results (which actually garnered only two finalists that year and resulted in both being declared the winners).
Just as Arne Duncan shaped the process dramatically when he chose his four criteria for “Race to the Top,” so too could the Broad Foundation have influenced urban schools if it had significantly rewarded (for example) project-based learning.
And Foundations do intervene/interfere big time, as you know. In fact, by NOT having a charter award for free-standing charter schools, the Foundation is sending an expensive signal. That’s their right, of course, but it ought to be looked at squarely…..
John
LikeLike
[…] the PBS Newshour, and he has learned quite a lot, most of which he puts into his forthcoming book, Addicted to Reform (out on August […]
LikeLike