Measuring soft skills

John Merrow school reform, Testing

(This post was co-authored with Arnold Packer.)

Reliability and Validity are the Alpha and Omega of testing. A test that is reliable can be counted on each time it’s given, while a valid test measures what it is supposed to. Tests that meet these two criteria are the gold standard of assessment..

Soft SkillsFor example, making someone swim 100 yards to test whether or not he can swim would be a valid and reliable test. If you sink, you flunk, and that’s true each time the test is given and is independent of who is doing the testing.

However, when teachers are trying to assess ‘soft’ skills, the waters get murky. How can we measure the ability to work with others, process information from disparate sources, communicate persuasively, or work reliably?

Continue reading →

Schools and Cyberbullying

John Merrow Below C Level, bullying, parenting, teaching

In late June Jan Hoffman of the New York Times explored the tough issue of cyberbullying and the schools. She led her provocative piece with an anecdote about parents asking their 6th grade daughter’s principal to intervene in a particularly difficult situation involving abusive and sexually suggestive email from a boy. They didn’t want to involve the police, and they knew the boy’s parents socially. The principal’s response was cut and dried: “This occurred out of school, on a weekend. We can’t discipline him.”

CyberbullingAt first I thought that was a legalistic, hair-splitting response—until I read about a principal who did get involved, was subsequently sued by the angry parent of the offending child, and lost. That’s horrifying, but it’s the reality.

My takeaway, however, is not that schools are right to split hairs and decline to get involved. Instead, I think we need some redefinition, some fresh thinking.

Continue reading →

Will national standards ever arrive?

John Merrow innovation, school reform, Testing

As I write this, close to half of the states have signed on to the draft of national standards, officially called the common core. Observers are predicting that well over half will be on board by summer’s end.

National StandardsThere’s a long way to go before we have genuine national standards in core subjects, and there’s no guarantee that they will be challenging enough, given the inevitable pressures to water them down.

And if we do develop worthwhile standards, some form of national testing is likely to follow.

The President of the United States is already on board for that. He said, “I believe we need some national standard education achievement tests—to be used only optionally when states and/or local school systems want them.”

Whoops, that wasn’t Obama; that was Jimmy Carter in 1977.

By the way, the public is on board. 77% of the public favors using national testing programs to measure the academic achievement of students.

Whoops, that was the Gallup Poll back in 1989. Continue reading →

Writing ‘Below C Level’

John Merrow Below C Level, Michelle Rhee, paul vallas, reading

In my work for PBS NewsHour over the past three years, I am most often asked two very specific questions: “Is Jim Lehrer ever going to retire?” And “What is your personal opinion of Michelle Rhee? Do you like her and what she’s doing in Washington?”

Below C LevelTo the first question my answer is always the same. ‘I hope not.” Of course I never answer the second question when I am asked, because it’s our job to report what we see happening, not express opinions or pass judgment. I do, however, have some thoughts on the subject, which you will find in Chapter 9 of Below C Level, pages 81-105. Yes, it takes 24 pages.

I spent five and one-half years writing Below C Level. The first drafts of many of the chapters were written on an airplane—I haven’t watched an in-flight movie for years—because my work takes me to distant places, and I have been living on the West Coast for the past eight years.

But, looking back with the first copy of Below C Level on the desk next to me now, I realize that the first five years were a walk in the park, relatively speaking. The last six months were without question the hardest part of the journey. During that time I rewrote every one of the 37 chapters. Once rewritten, it then had to find a place in the structure of the book, or go into the circular file. Continue reading →

Summer reading

John Merrow Below C Level, reading, summer

Well, it’s finally here, the proof copy of Below C Level. I looked it over and then went on line to give my approval. That one keystroke made the book available immediately.

Below C LevelAnd if you want to be the first in your neighborhood to have a copy, please visit the Below C Level website or purchase it online directly at www.createspace.com/3422460.

When you do that, you will be getting a pretty good read and also helping Learning Matters–because I am donating most of the royalties to the company.

What’s in the book, you ask? Well, It’s 432 pages before you even get to the index, 37 chapters covering everything from pre-school through higher education.

A wry friend suggested that, if I wanted to sell a lot of books, I should just write about all the people I’ve interviewed over the past 34 years. An option, he said, was to tell the truth about them. I’ve actually followed his advice (including the option); the index is 11 pages of mostly names. Continue reading →

What you didn’t see on television

John Merrow media, Michelle Rhee, paul vallas

My colleagues and I have spent the past week or more putting the finishing touches on the last installment of our reporting from New Orleans and the Recovery School District there. In all, PBS NewsHour will have aired 12 segments about Paul Vallas and the RSD, and we also produced three other post-Katrina (pre-Vallas) segments. (Watch the full Paul Vallas series here.)

Paul Vallas in New OrleansThat’s 15 segments, each 8-10 minutes in length, a total of 2 hours of television, roughly. You might be interested to know what went into creating those two hours. Each piece generally entailed three days of shooting, perhaps 6 hours of videotape each day. That 6 (hours) X 3 (days) X 15 (segments) = 270 hours in all.

Our monumental task–15 times over–was to then take that raw material and edit and shape it into a short segment that would tell some part of the story of the effort to transform what was easily one of the worst school districts in the nation.

We produced more than our reports for PBS NewsHour: Each piece was accompanied by as many as four podcasts, usually longer interviews with Vallas, State Superintendent Paul Pastorek, various Teach for America teachers, parents, and so on. (Listen to the podcasts here.)

We’ve been doing the same job in Washington, DC, chronicling the efforts of Michelle Rhee to reform the schools there. We’ve made as many trips, shot as many hours of video, and spent as many weeks editing. We’ll present our final chapter from that city later this summer.

Did we get it right? Continue reading →

Game-changing innovation

John Merrow innovation, technology

Those of you who look at these posts with any regularity know that innovation is an interest of mine. I’m a fan of Clay Christensen’s observations about innovation being far more likely to succeed on the margins (where few are paying attention), but these days the margins in education are wide—because only math and English matter to the bean counters!

Innovation is education’s only hope. Even if more stimulus dollars become available, that will only put off the day of reckoning, unless educators wake up and act.

Happily, some are. Here are three examples. Continue reading →

Thinking out loud about remedial education

John Merrow innovation, teaching, technology

I’m in Seattle, where about 250 people, mostly grantees of the Gates Foundation, have gathered to talk about increasing the rate of success in higher education, especially among low-income students. My role is to lead a conversation with three individuals who represent three different slices of the industry: public community colleges, for-profit institutions, and on-line universities.

The Foundation was kind enough to suggest some opening questions, such as “How can we serve more low-income students better, faster and cheaper?” and “What do each of your institutions have to offer?” and “What will higher education look like 5-10 years from now?”

Maybe it’s because I am still medicated after my recent knee replacement surgery, but I find my mind wandering to other, perhaps more provocative questions. Continue reading →

Solving a man-made problem

John Merrow reading, summer

One of education’s dirty little secrets is that schools give what they call their ‘end of the year’ tests about six weeks before the end of school. The school year is only 5/6th of the way done, but it’s testing time, and everything stops.

School's outThink about that for a moment, maybe put yourself in the shoes of a teacher or a student. If you’re a kid, the message is clear: the year is over! Time to kick back and relax. However, if you happen to be a conscientious teacher, you have to climb a big hill every morning and afternoon for the next five or six weeks, because you have to try to interest your students in what they know doesn’t matter.

Left unexplored is what’s being tested. Do these tests cover everything that the students are supposed to have learned, or merely 5/6ths of the material? If they cover everything, isn’t that unfair to those who are being judged by the results (students and, increasingly, their teachers)? If they cover just 5/6th of the course content, will that mean that many students will never get past, say, World War II in history?

Why this happens is no mystery: it’s done for administrative convenience, to give the test companies time to run the test papers through their machines, process the scores, and get them back to the school districts as early as possible in the summer. With so many tests and so few companies, the bottleneck is frightening, worse than the lines at the toll booths on a summer evening when everyone is driving back from the beach.

What makes this bottleneck worse are the mistakes that occur. Imagine if the toll collectors randomly collected different amounts from some drivers! That’s what happens when the machines begin their mass processing: they make mistakes! Continue reading →

One Good Turn…

John Merrow Testing

If you’re a reporter or any sort of education wonk, then you’ve been aware of FairTest, the shorthand name for the National Center for Fair and Open Testing. The folks at FairTest have been superb advocates, warning us of the dangers of mindless embraces of bubble tests and cheap standardized, machine-scored exams.

Fair Test

I believe we need FairTest more than ever. No Child Left Behind is still the law of the land and is likely to remain so for many months. As we move toward common standards, we need voices calling for multiple ways of evaluating schools, teachers and students. Those who believe that one score tells it all are sadly wrong, but they are also very powerful.

Sophie Sa, a friend of many years who is now FairTest’s Board Chair, reminded me that FairTest was created in 1985 “to fight against the misuses and abuses of high-stakes standardized norm-referenced tests … while promoting the use of multiple forms of authentic alternate assessments.”

I agree with Sophie that FairTest has been “the leader, the organizational glue, and the legs for individuals and groups concerned with the deleterious effects of standardized testing on students, particularly students in need; on schools and their curriculum; and on the kind of learning that emphasizes critical thinking and deep knowledge.”

Sophie and I go way back, to my own professional crisis back in 1994. Continue reading →