As always, remember that John’s book The Influence of Teachers is for sale at Amazon.
Forget cheating on tests for a minute and think about the concept of ‘teaching to the test.’ Just what does that mean? The usual line (which I have used myself) goes something like this: “It’s OK if it’s a good test,” and that may be correct. Unfortunately, most of the tests that I have seen are not ‘good’ tests.
Think about teaching students to write, and then testing their skills. Clear writing is important. Employers want to hire people who can write clearly, accurately and well — but learning to write takes time and requires rewriting and more rewriting, under the guidance of a good teacher. There are no shortcuts. However, our obsession with numbers subverts both teaching and learning. Teachers are told that their students must be able to pass bubble tests and write a lot of short so-called essays (usually one or two paragraphs!) There’s no time for reflection or rewriting.
Instead, students are drilled in the ‘constructed response’ process: write a declarative statement and then add three or four details to support a statement, such as: “I always use sun block when I go to the beach.” And so they follow the formula they’ve been given and produce something like: “I always wear sun block when I go to the beach because too much sun can cause cancer, and because too much sun will make me all wrinkled when I get old, and because cancer can kill you. My mother makes me use sun block too.”
That ‘essay’ would get a passing score because the student supported his statement in four ways. The teacher (or machine?) grading the ‘essay’ could simply count the supporting reasons. Everybody — teachers, principal, superintendent and school board — would pat themselves on the back, but is Microsoft, GE or Hilton likely to offer someone who’s been trained to write that way a job?
That’s what we are doing to our children. It’s only slightly hyperbolic to say that we are lying to our kids.
Cracking down on cheaters — which we should do — won’t fix our problem. Think about it this way: You are sitting in your living room when drops of water begin falling on your head. Clearly, you have a problem. If you move your chair, have you solved it? After all, you no longer have water falling on your head.
Of course not, because the problem persists, although now the water is falling on your living room rug. Suppose you get a large pot and place it where it can catch the falling water? Have you solved the problem? Of course not, because you still have a leak somewhere.
You get the point. I think it’s time for those of us who are attacking bubble testing and the intense pressure to ‘produce’ to back off and ask, “Where do we go from here?”
Unfortunately, we haven’t asked and answered that question in the past. Subverting the testing system is an old story that we don’t seem to learn much from. Remember Austin, Texas, where most of the school board was implicated in test score deception? How about that small town in Connecticut with its ‘miraculous’ test score gains a few years ago? Not miracles, just plain old cheating.
Sometimes the system aids and abets the deception, as in Florida, where a loophole in the state law allowed districts to counsel low-performing students to drop out to go into GED programs. By law, the districts didn’t have to count these kids as dropouts as long as they suggested the GED alternative, no matter that no one had to follow up to see if the kids actually enrolled.
How about the so-called ‘Texas Miracle” that turned out to be the ‘Texas Mirage?’ Houston had great test scores, and Superintendent Rod Paige eventually became U.S. Secretary of Education. Then we learned that an inordinate number of low-performing 8th graders were simply being held back, often for more than one year, because high-stakes testing didn’t begin until 9th grade. Some find the seeds of No Child Left Behind in that misadventure.
Atlanta may actually be the proverbial tip of the cheating iceberg because evidence that suggests major cheating has also occurred in D.C., Pennsylvania, Florida, Houston, Baltimore, Los Angeles and elsewhere.
Some consultants, test security companies and even the test makers themselves are licking their chops right now, expecting to make a lot of money designing what they will claim will be better defenses against cheating, because ‘firewalls,’ ‘fail-safe’ steps, ‘erasure detection software’, and other ‘technical fixes’ are a big part of the conversation. In fact, Education Secretary Arne Duncan told the Atlanta Journal Constitution:
“The technical fix is very simple, and they need to put that in place. The job for a new superintendent coming in after a crisis is to rebuild public confidence with absolute integrity, transparency.”
I respectfully disagree, because cheating is not the real problem; it’s a symptom of a larger problem, and the solution is not simple. Not by a long shot.
The problem in Atlanta, in D.C., and wherever else cheating is occurring proves Campbell’s Law, which states “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
Live by the test, die by the test.
We rely too heavily on the scores of relatively simple (and relatively cheap) machine-scored ‘bubble’ tests as the measure of educational accomplishment, and that invites deception, cheating and criminal behavior.
So where do we go from here? Well, we aren’t going to ‘get rid of testing,’ that’s for sure. Anyone who wants to throw out that bath water ought to recall the New Orleans high school valedictorian that could not pass the Louisiana state graduation test, despite being given multiple opportunities!
Nor is it enough to endorse “multiple measures” of achievement. It’s more complicated. We have to ask ourselves what we want young people to be able to do upon graduation and figure out how to teach and encourage those behaviors. Then — and only then — do we figure out ways to measure them.
What if we were to ask large employers like Michael Dell, Steve Ballmer of Microsoft, Carol Bartz of Yahoo, the heads of Hilton, Hyatt, Avis and Hertz, Wendy Kopp of Teach for America, Steve Jobs, Jeffrey Immelt of GE, the provosts of some major universities, top advertising agencies and so on what they look for in potential employees? What would they say?
Or maybe you hire people for your company. What do you look for?
Life is not all about work, of course, so we ought to ask what we want our youth to be: good parents, concerned citizens, informed voters, discerning consumers, and so on.
Then let’s figure out what sort of school-based experiences teach or sharpen those skills and attributes. My hunch is that group activities and project-based learning will figure prominently. I think we will be reminded of the truth of the late Ted Sizer’s observation that “Less is more.”
Tests drive public education right now. But what should be driving the enterprise are agreed-upon goals that come from the real world.
Where do we go from here? That’s up to us, isn’t it?
The Merrow Report 
Well I’ll open the bidding. Yes, we do know what employers want, and what our democracy needs. Takes up one paragraph in:
“Nine Lies about Academic Achievement that Parents and Teachers often Seem to Believe—but Don’t Really.”(http://bit.ly/jjLkL2)
It is also what will get you into college.
What’s happening is that we are in transition from a pyramid model of society where life is a race to the top and academics will get you there, to the world we have been living in for at least a generation. Getting ahead has been replaced by finding your “hedgehog concept”: (Jim Collins) and making something of your unique self. Hundreds of schools have abandoned the Pyramid Model and moved on, graduating millions of smart, self-confident, collaborative learners and leaders. BUT there are still thousands of archaic schools abusing 10s of millions of young people.
We have to changed the game of school from “Get the right answers” to “Work with Others to Investigate Interesting Questions.”
Thank you, again, John for pointing us in the right direction. We simply have to see past the politically based, ill-conceived debate about testing and get down to what education is. We have to change the very culture of schools to conform a new emerging global culture.
LikeLike
Teaching to any test or, more aptly, teaching test-taking strategies is a waste of time and real learning opportunities. One researcher, Louis Volante (Canadian Journal of Educational Administration and Policy, 2004) clearly states that, “Teaching to the test not only reduces the depth of instruction in specific subjects but it also narrows the curriculum so that non-tested disciplines receive less attention during the school day.” My own in-class data is even more profound.
Before creating a Results Only Learning Environment, I used to spend countless days teaching test-taking strategies and offering one test model question after another to my students. In the last two years prior to the ROLE, 26% of my students failed the test (one of the years, 33% failed).
The first year of results-only learning, which eliminates all teaching to the test, practice testing and teaching of test-taking strategies, my students’ test scores increased nearly 11% over the past two years combined. Consider the impact of this fact: for two years, I tried every method that my school made available — practice tests, model test questions, constant emphasis on test-taking strategies, and online tutorials — only to see a passage rate of just 74%. With all references to the test gone, nearly 85% of my students passed!
Results-only learning emphasizes year-long projects that embrace all learning outcomes. Students gain a new thirst for learning, because of the autonomy they are given to choose how they demonstrate mastery and the complete elimination of traditional methods.
Teaching to the test and the test itself must go.
LikeLike
As I write you can see Joel Klein sitting behind Rupert Murdoch on your tv as Parliament has at him. For those of us who had to watch Klein’s career close up the sight is unsettling but not surprising.
We who care about education should not lose sight of the fact that for many who have been making education policy for the last number of years it has always been much more about power than about education. Power is the end education just the means.
To put it another way, the real tragedy is not in our current policies but in the fact that we have so readily surrendered democratic checks and balances for pie in the sky scenarios of ‘magic bullets’ (charters, slim curriculums,lotteries,choice, mayoral control). So we must not fall into the trap of being sidetracked by details. ( A favorite trick of credit card companies.)
Or to put it still another way, here in sophisticated New York one day in May of 2005, in the midst of a successful mayoral campaign, Michael Bloomberg stood in front of P.S. 33 in the Bronx and bally-hooed a near 50% increase in 4th grade state reading scores-without a trace of irony . I doubt that even one head shook that day in stunned disbelief.
If memory serves, Joel Klein was there that day, too.
LikeLike
You get the point. I think it’s time for those of us who are attacking bubble testing and the intense pressure to ‘produce’ to back off and ask, “Where do we go from here?”
#1 We need put less emphasis bubble only tests. They serve a utilitarian purpose because they are quick, cheap and easy to grade but
1) they separate the teacher from the student because the teacher doesn’t not personally correct the work
2) they are easier to cheat and fake on.
3) the emphasize superficial learning and guessing.
Bubble tests and bubble reviews encourage students to go through the motions of learnng. Learning can be hard. You have to look things up. You will make mistakes. You should learn from your mistakes. Finland does very well withouth bubble tests en masse. Most of their tests are teacher made.
To me the ideal test has a very easy portion for the lower level students and then gets increasingly harder. No “A’ student should be able to get an “A’ in a literature or social studies who cannot define key terms and answer a simple essay in a clear and literate fashion. Also high school students should be able to do classwork and homework and keep them in an organized notebook. I grade homework and the notebook. Students who fail to do their homework five times in a quarter find their grade lowered 10%. Students how fail to do their homework ten times find their grade lowered 20%. It is a sure fire way to find cheaters. Cheaters usually try to get by on cheating not doing classwork. It is possible that a cheater might have gotten a D in my class but not an A or B. By grading all my students’s work and checking their notebooks at least twice a week get to know them. I also grade them on class attitude and participation. Frankly, I am more impressed when someone orally explains a historic even and connects it to something else than if someone marks a “snake egg” on a scantron ten times. Did they really know it? Did they copy it? Did they guess? By quizzing students orally I learn a lot. My favorite oral quiz is of the student who has a 100% which surprises me. I ask them to sit in from of the class while I quiz them on the quiz they just got a 100% on. If they cannot answer a single question correctly I mark their test NC (no credit). A classroom teacher should be able to mark a test invalid if he or she thinks it is an invalid test.
Concurrently, sometimes I give a confusing question as a bonus question and accept two answers. Sometimes I invalidate the entire quiz or test or part of it. The classroom teacher has to use his or her judgment and operate with the utmost integrity.
LikeLike
Richard, I appreciate that you want to de-emphasize standardized tests, but I’m troubled by your alternatives to assessment. You seem to be anti-multiple-choice test, but not only are you pro homework, you also want to punish students by lowering their grades exponentially, when homework is unfinished. This is a vicious cycle that makes for very poor education, in addition to making students hate learning.
There is virtually no empirical data to support the use of homework. In fact, Alfie Kohn’s unparalleled research, demonstrated in his book The Homework Myth, proves time and again that homework does not improve achievement and that it is, in fact, damaging to students.
Cheaters cheat, due to the pressures brought on by letter grades, oral quizzes and GPAs. Remove these, and cheating will quickly disappear.
We need to create a thirst for learning, which eliminates the need for testing, homework and grades. Well-designed year-long projects that integrate student choice in how they demonstrate learning outcomes, including a wide array of Web 2.0 and social media tools, is the best way to create the aforementioned thirst for learning. Narrative feedback between the student and teacher, followed by project improvement, can easily replace testing and grades.
It’s time for change in American education, but most of the evidence we have here and around the blogosphere demonstrates that we aren’t there yet.
LikeLike
It has been alluded to in the commentary and comments: when thinking important goals for “teachers,” they need to be (1) facilitating students’ abilities to learn; AND (2) providing an environment that encourages autonomy, mastery, and purpose (in the words of Dan Pink in “Drive”) – increasing motivation levels for learning AND for use of knowledge and problem-solving skills to address real problems.
Until the focus of education (and assessment of it – I’d recommend portfolios as the approach) is changed to such an approach, education will not be optimized for students. AND, oh yes, employers will not find enough of the hires they seek and our country will not maintain its level of leadership in the world.
LikeLike
i appreciate some of you suggesting alternatives, because ‘where do we go from here?’ matters. Right now we are immobilized and polarized. I worry that the July 30th rally in DC will be more of the same, lots of talk about what they are opposed to but little in the way of genuine suggestions and alternatives.
Reagan used to say ‘trust but verify’ about the Soviet Union. We need a ‘trust but verify’ approach in public education.
More about that later
LikeLike
John,
Your focus on some of the essential issues is uplifting. We need a national campaign dedunking ‘teaching to the test’. As an educator, with graduate level degrees from a well respected teachers college, nothing in any of my coursework and practical training support “teaching to the test”. In fact, doing so invalidates the test (if it has validity) and undermines the prupose of the test, regardless of how good the test may be. In addition, a test is only one assessment tool, and when a test is given it is only one measure of the student’s performance on whatever was tested. It is impossible to accurately deduce the student’s overall achievement based on a single test, regardless of how good it is. And a good evaluator would want to examine/compare other information that supports or does not support the outcomes of the test.
Teachers must know the purpose of the test and what it is designed to measure in each section or content area. They need such information in order to be good evaluators of their students’ test results. For example, the item that you gave, “write a declarative statement and then add three or four details to support a statement”. If such an item merely requires the student to follow the formula, then it might be a good test item in a section on following directions. A test should be designed to measure knowledge and skills in areas of instruction that have been taught. If content and skills areas have not been adquately covered in instruction, then high test scores are unlikely.
One problem in education that is very alarming to me is that ASSESSMENT has been replaced by testing, and the role for testing seemingly has wide acceptance. Our federal programs (NAEP and others) should take the lead in correcting this problem.
“No Child Left Behind” is a wonderful concept. Fund it to implement teaching and leanring strategies/approaches that have been tried and have successful outcomes; start in the classroom with the students and teachers. Fund NCLB to create 3-tiered programs for: 1) student advancement to grade level, 2) maintenance at grade level and 3) exceleration beyond grade level. Fund NCLB to do real, valid, reliable ASSESSMENT of progress (at intervals within each tier) using both empirical and statical outcomes.
Preparation for work is certainly a very important product of education, and your point about writing skills is only one of many examples of the role it plays in the workplace. Furthermore, education contributes to every role in our lives. The large number of very capable college freshmen needing remedial writing and math is an example of an educational role not fulfilled.
There is much more that I would like to say,but my time is limited right now. Thanks for your focus John Merrow.
.
Nona Smith
LikeLike
Beware the Jabberwock. Testing is but a symptom, not a cause, and it’s a symptom of an industrial vision of what’s not really an industrial enterprise. Simple measures – like profit or loss, gain scores or failure rates – belie the complexity of what goes on in schools and colleges. And fighting that simplicity works as well for progressives as fighting capital did for Marx: you build a little steam, find a Lenin or Trotsky, blow up a few places, but it slides back into the Jabberwock.
The organizers’ tool is to use the weapon of the badguys against them. Find a better measure and make the case that theirs is fine, but only for a part of the picture. And there are many, many alternatives. One of the most banal but critical is attendance: do kids go, are they on time, and do they maintain their attendance rates over time. Banality – like testing – has its place.
Other measures are what they do with what they learn – both in and out of school. That used to be difficult to collect, but, with tech and facebook, social networks and the fuzzy line twixt work and play, it’s a lot easier. If they can brag about skills out of school, and if those skills actually improve in the course of school, well, correlation may not be cause but it’s at least more than conflict.
My preference, which I’ve blathered about here, is portfolios, particularly if they’re organized with some standards like Packer’s “soft skills” or Sternberg’s “Wisdom-Intelligence-Creativity-Synergy.” Organizing achievements against some standards infuses those portfolios with reflection, and it’s the reflection that counts more than the numbers of a test, days of attendance, or richness of the network. Kids who are mindful – in Ellen Langer’s term – or reflective are loads more “qualified” for jobs, careers, college and success, since, among other things, they learn how to measure that success in terms that have meaning to themselves, and often to family, friends, or community.
It really is that simple. All the other metrics are full of sound and fury, signifying nothing, and, by the way, told by an idiot.
LikeLike
After six years of teaching in the toughest trenches in Boston, the same things I learned in my M.Ed. Program apply, but are rare, and while most teachers know this, they are not given enough voice to cause change. Children beyond third grade need project-based learning that is grounded in essential questions that are of concern to them, their communities and the world at large. Social skills curriculums should be in place and embedded in everything we do. Assessment should be to inform teaching and assess individual progress, and should not be used to evaluate teachers, administrators, schools, or school districts, to eliminate the anxiety, unfairness, discrimination and deception this causes. Also, those employed in our schools must be treated in a way that encourages participation in creating change. Both teachers and students need a system that understands human motivation and self-determination theory.
Finally, we have the ultra huge burden of having to address the diverse language needs of our population. I’m wondering about this a lot. This is a giant drain on our resources. How can we be all thing to all people? While I adore diversity, language difficulties in our classrooms are diminishing progress.
LikeLike
After six years of teaching in the toughest trenches in Boston, the same things I learned in my M.Ed. Program apply, but are rare, and while most teachers know this, they are not given enough voice to cause change. Children beyond third grade need project-based learning that is grounded in essential questions that are of concern to them, their communities and the world at large. Social skills curriculums should be in place and embedded in everything we do. Assessment should be to inform teaching and assess individual progress, and should not be used to evaluate teachers, administrators, schools, or school districts, to eliminate the anxiety, unfairness, discrimination and deception this causes. Also, those employed in our schools must be treated in a way that encourages participation in creating change. Both teachers and students need a system that understands human motivation and self-determination theory.
Finally, we have the ultra huge burden of having to address the diverse language needs of our population. I’m wondering about this a lot. This is a giant drain on our resources. How can we be all things to all people and expect to get ahead? Language difficulties in our classrooms are diminishing progress. We need to build the vocabulary and background knowledge of our children very early through various media, programming and technology that is available to all families.
Thank you for your leadership in ensuring SOS day contains specific action suggestions.
LikeLike
What’s also sad is that middle and upper middle class schools, which often don’t have to worry as much about test scores, are doing more of what Merrow suggests at the end of the article. They’re doing research. They’re doing collaborative projects. They’re organizing and presenting information within a variety of formats.
Since test scores tend to correlate with students’ socioeconomic levels, schools in lower-income neighborhoods are hit much harder by AYP pressures, threats of school closure, and by the instability and high staff turnover that these pressures bring with them. They are often forced to teach to the test because they are under so much pressure to bring up their scores in a short amount of time. So students at those schools read short excerpts, fill in bubbles, and write formulaic responses. And in the process they become even less prepared for the professional jobs that their schools are supposed to be training them for.
LikeLike
I have to show some thanks to you for bailing me out of this situation. Because of [url=http://www.guccihoshionline.com]グッチ 財布[/url] searching through the internet and getting ideas that were not powerful, I assumed my entire life was done. Living minus the approaches to the issues you have resolved by means of the website is a critical case,[url=http://www.guccihoshionline.com]グッチ バッグ[/url] グッチ as well as the kind that could have in a wrong way damaged my career if I had not discovered the blog. The understanding and kindness in playing with almost everything was very useful. I’m not sure what I would have done if I had not [url=http://www.guccihoshionline.com]グッチ 財布[/url] グッチ 長財布 discovered such a thing グッチ 長財布 like this. I can also at this time relish my future. Thanks very much for the reliable and results-oriented help. I will not hesitate to propose your web sites to anybody who would need care about this issue.
LikeLike
How can you acknowledge “All games die e 10 million” when it’s unlikely that any MMORPG games have got that many subscribers…
LikeLike