Proof that teachers matter

On Sunday the Los Angeles Times published a story that has created a small firestorm in education circles. Three reporters documented the effects that teachers have on their students’ test results.  And they named names, so that now the world knows that students in John Smith’s fifth grade class start out ahead but lose ground as the year goes on, while Miguel Aguilar’s fifth graders follow the opposite trajectory: they do poorly at the start but outscore Mr. Smith’s students by year’s end.

Over seven years, John Smith's fifth-graders have started out slightly ahead of those just down the hall but by year's end have been far behind. (Irfan Khan, Los Angeles Times)
(Irfan Khan, Los Angeles Times)

Those are just two of the names the Times printed, and the union is furious, calling for a boycott of the paper.

But is it wrong to speak the truth?  Is it wrong to call out ineffective teachers?  That’s the debate going on, with even the Secretary of Education weighing in on whether it’s appropriate to make the names public.  (For the record, Secretary Duncan approves.)

Let’s be clear about one thing: the Times is most definitely NOT breaking new ground when it tells us that some teachers are effective and others are not.  Every parent knows that, and savvy parents lobby for teacher so-and-so for their children.  My wife and I were at a block party just last night where the subject came up. Earlier in the day I was bicycling with friends, and one woman described how hard she had worked to make sure that her twins had a certain math teacher in middle school.

I firmly believe that just about everyone in any school can tell you who the really good teachers are in the building.  Whether they will tell you is another story, perhaps, but everyone knows who’s good and who’s bad.

Reading the Times piece I was reminded of a paper that Dan Fallon, formerly of Carnegie Corporation, shared with me some years ago. It’s a powerful demonstration of the influence good teachers have.  I have a chapter in my new book, Below C Level, about this as well.

I applaud the Times for bringing this to the forefront.  I worry that it could be a step backward if it merely heightens the significance of scores on bubble tests, but that’s a risk worth taking.

One phrase in one sentence early in the piece is the key, in my view: “year after year, one fifth-grade class learns far more than the other down the hall.”

And then:
In Los Angeles and across the country, education officials have long known of the often huge disparities among teachers. They’ve seen the indelible effects, for good and ill, on children. But rather than analyze and address these disparities, they have opted mostly to ignore them.

That’s the central point: the adults in charge have known of the damage that some teachers are doing—and have done nothing, or nothing effective anyway, about it.  That’s the high tolerance for mediocrity that I find alarming, and that’s what must be addressed, and soon.

Of course it’s possible.  Two years ago we watched the chair of the math department in a DC high school going over student scores with his faculty. He was able to pinpoint which teachers were apparently not doing a good job of teaching particular concepts (quadratic equations, for example) because he had student results matched up with their teachers.  His response was to offer those teachers new strategies and approaches, to give them opportunities to get better.  How can anyone find fault with that?

The next step, of course, is to remove those teachers who, for whatever reason, do not improve.

So rather than boycott the LA Times, I say we should all subscribe.  And we should turn up the heat on administrators who refuse to set  and maintain high standards for their teachers, and on unions that don’t work hard to give teachers opportunities to be excellent.  Your thoughts?

FURTHER READING

Who’s teaching L.A.’s kids? [LA Times, 08/14/2010]

The Amazing Miss A and Why We Should Care About Her [Dan Fallon’s 2001 paper]

102 thoughts on “Proof that teachers matter

  1. Absolutely nothing wrong with naming names – as long as the evidence is defendable. Let me describe a scenario requiring significant study: teacher A knows that students have to make significant appropriate efforts on the varied and considered assignments she makes for the students to learn for retention and ease of use. Whether the students are “lobbied” effectively by parents or just assigned, there will be likely fall into two groups: the group that buys in and assesses well on any assessment efforts; and the group that complains / doesn’t understand and doesn’t measure up. Most assessments will be bimodal requiring deeper feedback to understand. Quick review of numbers could lead to any decision. Only further investigation will show if the teacher is assisting any making the effort to improve or expecting all to improve because she has good materials / assignments but is a lousy facilitator.

    Th

    Like

  2. Once again success is defined solely by a standardized test – this is not the Learning Matters I have grown to know and love – does the Gates foundation have anything to do with your narrow definition of what learning and teaching should be? I would really like to know, because I am beginning to wonder about where to get my news on education policy.

    Like

  3. My point is that, long before the dominance of test scores, savvy people knew who the best teachers were, and who the poor ones were. The system has never done much to try to change that, and it should. If administrators know that my students rarely learn to multiply fractions and I won’t or can’t improve, then I should lose my job. Administrators who don’t act should lose their jobs. Not sure how Gates comes into this. Please explain

    Like

  4. It is not wrong to identify, assist, and when necessary, remove ineffective teachers–or administrators for that matter. There seem to be different issues in this situation, however.

    One is what conclusions are being drawn from these test scores and are those conclusions valid. Those who know about testing and measurement know why the answer to that is not as simple as it might appear. When I taught a tested grade level and subject for many years, I was never afraid to have my test scores published; we are public employees and this is data to which parents should have access. What mattered to me was having the opportunity and the skills to look beyond the numbers and explain to parents what those numbers meant for each child. The test says Jamika is weak in grammar, but Jamika has written several assignments for me with no major grammatical flaws. The test says Ben is an exemplary reader; Ben made a tic tac toe pattern on his answer sheet (true stories). If I simply send their numbers to their parents, I may be giving a false impression of the student’s real performance. My concern here would be LAT might be doing the same thing to some of these teachers, if all they are using are the test scores for evidence, so I’m not as sure as you are about the wisdom of their journalistic decision to name names.

    Another issue, which you rightly raise, is the question of leadership. Are teachers and their work being evaluated on a regular basis? Is that evaluation effective? What steps are taken when a teacher is found to have areas of weakness and when are they taken? How well trained or experienced is the administrator? How often is the administrator evaluated? What supports are in place for him/her? You’re right; before the dominance of tests, it was possible to know the various levels of teacher and administrator quality, and we in education have tolerated the dance of the lemons for far too long. But I don’t see how the public humiliation of students and their teachers will help get us to solutions of the real problems.

    Like

  5. Years ago the only solid test information to me was the breakdown of my AP students in Spanish or US history. I used that information to become a more efficient teacher because it told me things I did not know. For example some of the best students had the strategy of doing their weakest essay first 13%, then their best essay 13% and then turning as much time as possible to the DGQ 23% . These students got 5 on the AP tests and students as able and with better grades got 4’s because they had to rush their DBQ. I found out it was important to practice 80 mc questions in 55 minutes often to simulate the AP test.

    Today we are using standardized tests as dipsticks for ALL STUDENTS and we are analyzing the results. We have begun to reveiw and study for the CST’s as if they were AP tests. We have the THINK GOLD program to give students rewards and honors for doing well on the CST’s And the result has been our AP has gone up consistently for the past five years to a respectable 647. A teacher needs to be brave. I have all the English learners. I have over 25 years teaching experience. I am not afraid to have the lowest scoring students. I have had the lowest scoring students for each of the past five years. But guess what? On average these students show growth every year.. Our graduation rate is improving. We have the best English reclassification rate in our country. Kids are learning English. For the first time some EL students are scoring Above Basic and some Far Above Basic. Others are Below Basic but fewer are far below basic. Monday i will be teaching six perids of ELL students including two periods or ELL Level 1 (some do not even know the English alphabet). I am not afraid. I know it is a challenge. I know in many ways I have the hardest job -a job few people want. But if you do the right thing and have the right materials you can make a difference in kid’s lives.
    We can’t just teach AP kids. I loved teaching AP kids but let me tell you -they were the easiest kids in the world to teach. I went 12 years without a single D or a single referral. But let me tell you I find talented kids in the ELL Level One classes too. Many become FEP (Fully English proficient) and some go on to college. But I will tell you something. Some do not make it just now. Success may come later; perhaps even in later generation. But I always tell the kids that if you leave school with oral and written fluency in English your school years have not been wasted. And I tell them they are young and they can finish their education at a later time IF THEY MAKE SACRIFICES and IF THEY ARE MOTIVATED TO DO IT.

    Standarized tests are not everything. They are just dipstick and a narrow measure. But like my bloodpressure test at the store vital statistics gives us something target and can help us improve as teachers. Don’t teach to the test. Teach beyond the test. But above all do not concern yourself too much witht the results just do your very best and seek betterment. The real test as I tell my students is life.

    Like

  6. Arne Duncan and you are right. Names should be named, and remedial action should be taken. It is long past time for the teachers’union and its supporters to stop blaming the children, the parents, the funding and the tests. Every effective organization knows it has weak links that need to be strengthened or replaced.

    Like

  7. I need to take a deep breath and explain my concerns about the emphasis of standardized tests as a way to measure teacher effectiveness. Yes, undoubtedly there are differences in the quality of teachers – I’ve witnessed this as a teacher and as a parent of school age children. Yes, ineffective teachers should not be in teaching. As a matter of fact, individuals who do not exhibit intellectual and pedagogical ability should never be allowed into the profession. My concern, and it is a strong concern, is how we measure “effectiveness.” What makes a good teacher? This is an important question that I believe must be answered before we come up with a measurement for it.
    I am a third grade teacher and I worry that my answer to this question does not jive with Arne Duncan’s and many others who make education policy decisions, and I worry about how this conflict will impact the profession I love so deeply. I became a teacher for the reasons Sam Chaltrain gives in his lovely blog – to help students “unlock the mystery of who they are by acquiring the skills and self-confidence they need to be seen and heard (at college, in their careers, and as citizens in a democracy) in meaningful, responsible ways.” Yes, in part this goal can be measured by my student’s growth on standardized tests, but SO SO SO much of what I do in my classroom, and what I do outside my classroom to support other students, parents and my colleagues, can not be reflected on one-shot quantitative tests.
    I fear that if my role of a teacher is defined simply by standardized tests scores (such a narrow view of what I do) I will have to abandon my career before I relinquish my values. Please do not think that my students have performed badly on these tests over the years! Quite the contrary! My scores have been good, but I am unwilling to have my professional ability valued on such a meager scale. Review my scores but also review my portfolio, survey my students and parents, come and observe my lessons, my classroom environment, my commitment to my school and my drive to help other teachers excel! Etc. Etc. Unfortunately such a well-rounded review of my professional performance is neither as simple nor cheap as standardized testing.
    What I do is so much more than prepare children for a test. However, if there is a chance that my picture would show up in the Boston Globe, I am afraid that I could be driven to take a more “Kaplanesque” approach to teaching. If I am being paid to increase tests scores – shouldn’t my focus be on test scores? I fear it would be – and then I would have to leave teaching.

    PS I worry that with all its money, the Gates Foundation (although in many ways commendable) is helping foster this narrowing focus of teaching.

    Like

  8. I wish I had stressed with more effectiveness my point of agreement. I am NOT endorsing simple bubble tests or simplistic value added measures. I write about their limitations at some length in my book. My point is that ineffectiveness is tolerated, even when ‘everyone’ knows which teachers are good and which are not. I hold administrators responsible for the most part, not union, and there I take issue with my friend Steve Buckley. Every outrageous contract issue was agreed to by a school board, so rationally one must hold boards equally accountable.

    Like

  9. The saddest part of the LAT’s sudden release of teachers’ VAM test scores is that it comes at the very moment when the Los Angeles USD is initiating a three-year teacher effectiveness reform initiative, which needs understanding and support by all stakeholders, including teachers. LAUSD is trying to do it right. By thrusting this mean-spirited series of articles in teachers’ faces and adding to the “blame the bad teachers” politicized agenda, the LAT has done a disservice to precisely the problem they want to expose. The district knows teacher evaluation needs reform; the district has begun addressing the problem. Teachers did not create the problem, and teachers deserve to be a part of the solution. However, alienating teachers does no one any good.

    Like

  10. I taught in the elementary classroom for 42 years, and am now “teaching” teachers. So yes – I think I know what I’m talking about. Of course there is a difference in the effectiveness of teachers, and yes, some should get the ax. But as you suggest, it is the principal’s responsibility to make that decision. If the principal wants to use test results as ONE portion of the “evidence” against a teacher, fine. It is, however, unconsionable for a school district to give the news media information that makes it possible to link teachers’ names with test scores because test scores only tell part of the story.

    Like

  11. Neither Arne Duncan nor John Merrow have said that test scores should be the only factor examined when reviewing the effectiveness of teachers. Having been married for more than 30 years to a wonderful woman who has just retired as an inner city teachers with special need kids, I don’t need to be reminded that test scores don’t tell the whole story. Having said that, I think examining how much progress students makes should be one part of the assessment of students. Another part should be whether students think the teacher is fair, treats everyone with respect, and has high expectations for all. Another part should be whether families believe the teacher reaches out to them. Another part of an assessment should be a portfolio that the teacher puts together describing key accomplishments. It happens that I helped start and worked for 7 years in a k-12 DISTRICT public school that used such an assessment system for many years.

    Like

  12. John – I’m a little surprised. The problems with VAM methodology are well known. The problems with the tests themselves are well known. It was incredibly irresponsible of the Times to report using the labels “effective” and “ineffective.” The data simply cannot support those evaluations. Cannot. Support. Those. Evaluations. What about tutoring and pull-out support effects? Transience and absence effects? All of the unidentifiable factors that cannot be controlled for? The lack of an adequate sample size? The lack of random assignments?

    The Times reporters attitude as I heard one of them describe it on Marketplace (PRI) was essentially that the National Academies and others say that these data aren’t strong enough to make evaluations that lead to administrative action, but since we don’t take administrative action, we can do the evaluation. Wow. That’s hubris. It’s as if someone had some data on a medical procedure correlating to a medical outcome, and the medical research community says that they haven’t proven a cause/effect relationship so they won’t draw conclusions about the procedure; then journalists say that, well, the data is there, and it’s logical to think it’s a cause/effect relationship. We don’t care if it can be proven or not.

    I also have to tell you as a school insider that parents do not always know who’s effective or ineffective. I have seen examples of teachers who have a particular reputation, good or bad, and when their former students land in my class, I can see the strength or weakness of the students’ preparation in ways that may not correlate to that reputation. It is so complex, and every attempt to oversimplify it, as the L.A. TImes has done, or to defend what the L.A. Times has done, does a disservice to our students, teachers, and schools.

    Like

  13. John, that’s a really nice job of sorting through the red herrings and focusing on what we know and can act on.

    Just yesterday my wife and I were having a discussion that with a friend who at first disagreed with us. He is frustrated that as a rule, teachers have not been subjected to any real performance assessment. We agreed that we shared his frustration, but argued that using test scores to assess teachers is too dangerous to support. Not only do they measures only narrow slices of what we expect teachers to do, but there also are serious challenges to the construct validity and reliability of many of the most-used tests.

    My wife is an attorney now, but before that she spent a very rewarding stint as a special ed teacher. She said exactly what you said: We all know who the good teachers are and who the bad teachers are. We just don’t act on what we know.

    Being a researcher as well as the executive director of an education honor society, I chimed in with the observation that when you are talking about compensation and careers, “everyone knows who the best and worst teachers are” isn’t good enough without very astute administration. Even with astute administration, only rarely will it scale up beyond the beyond the building level. Nonetheless, “everyone knows who the best and worst teachers are” certainly is one of many things that we in education already know or know how to do, but choose to ignore anyway.

    The three of us agreed that actually, we share the same frustrations and agree on almost everything. Your blog reinforces that outcome.

    Thanks for putting the spotlight on the administrators who, indeed, are in a position to act effectively on “everyone knows who the best and worst teachers are.” I hope you can generate some enthusiasm for efforts to make that assessment measure firm enough that it can be used more widely than it is.

    Like

  14. Mr. Merrow,
    For a man with only two years of public school teaching experience, you pretend to know all the answers about what is best for education. I will say that you, sir, are a charlatan. I will also say that your lack of knowledge has nauseated me on more than one occasion. And finally, I will say that your insipid remarks about the effect that teachers have on students’ test scores have done a lot of damage to teacher morale and to education itself.

    I would like to point out that during my 35 years as a teacher in Minnesota’s public high schools, I have noticed that teachers such as you who cannot function in the classroom will stop teaching and get training to be a principal or a superintendent; or, thay will get into a position where their “expertise” will allow them to give advice to other teachers.

    I would like to ask you to give me equal space to rebut your remarks about the effects that teachers have on students’ test scores.

    Maybe, for once, you’ll learn something.

    Joseph Legueri

    Like

  15. Well, there it is, John. In your own words. You don’t like bubble tests or simplistic value added measures as a way of judging schools or teachers, but you applaud the Times for using them and for outing teachers of kids who score low on those bubble tests. That’s just like Arne Duncan, who recognizes the problem with standardized tests, sets aside $360 million for the development of new, better tests, but then goes ahead and bases his whole reform on the current, bad tests. He uses them as sole high stakes measures that determine teacher pay and even as the tool to determine whether or not teachers and principals will lose their job. This is the unethical nature of the current reform. You should hold your applause.

    Like

  16. What we really want is vastly improved instruction.
    So instead of chastising some teachers, let’s provide teachers with GOOD textbooks and really good instructional coaches.

    “Here’s a little math problem: In 2005, just 45% of the fifth-graders at [inner-city, Title 1] Ramona Elementary School in Hollywood scored at grade level on a standardized state test. In 2006, that figure rose to 76%. What was the difference?

    But there is another, more intriguing answer: The difference between the two years may have been Singapore math [textbooks]. “
    [Beginning of “At L.A. school, Singapore math has added value” [The Los Angeles Times, March 9, 2008]

    Actually, credit goes to the GOOD Singapore math textbooks and to really good Professional Development from math professor Yoram Sagher, who is mentioned later in the article.

    Like

  17. Everyone knows that the best teachers are Republicans. We have to get rid of these liberals that are ruining the education system. We need less government interference with our schools and more choice of which schools we send our children to. That includes choosing teachers that think like “us.”

    We don’t need standardized tests, we need litmus tests!

    Like

  18. I am totally behind John and the LA Times on this. His claim is undeniable: we all know who the better teachers are. I just fail to understand why teachers should get a free pass on accountability when the data is public data. How is it different from a player hitting .167 for his team – in the paper, every day? What bugs me about the lack of accountability is that another generation of kids will be lost. This is not about comfort level of the adults, it’s about a kid’s right to a good education. It should NOT be Hobson’s Choice – here’s your teacher, tough. That only works for the comfort of the school scheduler and the teacher, not for the kids. It is not enough for adults to have good intentions. If kids could vote with their feet, schools would be better – just as is true in all other walks of American life.

    When you consider that we hire teachers without making them teach and show video of their teaching the problem is compounded. Worse, most teachers are afraid of student and parent feedback and rarely solicit it. How in the world will you get better, then? (I worked with a guy who EVERY FRIDAY handed out index cards and asked his 9th graders to write: what worked for you this week? What didn’t?) We know from Goodlad’s studies and the national study of student engagement that the most boring methods are the most common methods. Why is this so hard for teachers to grasp and deal with?

    This is, as John says, the supervisor’s problem. It is the supervisor who defines the job, monitors whether the job is being done, and takes action when the job isn’t being done. Every great building Principal I have ever seen – and I have seen many – makes it their business to be in classes EVERY DAY and puts pressure on bad teachers to get better. Yes, of course, offer PD. But at a certain point, the weak ones who don’t improve have to be counseled out. And great admins. do this well – with no loss of dignity on either side.

    Test bashers: read my March 2010 article in Ed Leadership. Many of your laments about testing are misguided. The state test is typically MORE valid and rigorous than your own. More importantly, this is a study of value added over time, so it is a fair measure of teacher ability. A knee-jerk test bash before looking at the data is not warranted.

    Like

    • Grant Wiggins,

      I would like to take issue with a few points you make here. First, most of the best teaching I have done and most of the best teaching I have observed in my time as an instructional coach would NEVER make for a good video. To insist that a video demonstrating a teacher’s teaching style be a litmus test for hiring a new teacher would completely eliminate many of the best and most effective teachers from ever finding a place in this profession. The best teaching does not come from a lectern or in front of a SMARTboard, it comes from building relationships with students, discovering what present and past experiences have shaped their understanding and guiding them toward new experiences and encounters with materials that will provide optimal growth. Moments of teacher influence in these situations are difficult if not impossible to plan as they come about through a teacher’s responsiveness to what their students are doing. It is not an easy thing to do and it certainly cannot be scripted or reduced to a formula.

      Another problem I have not heard anyone mention in this whole debate is how can you have a test, be it a state mandated test or teacher drafted exam, that tests a teacher’s effectiveness when there is no agreed consensus on what the purpose of school is? The batter who averages only .167 and has his average printed in the paper at least knows what the purpose of baseball is. What is the purpose of school? Is it to prepare workers and consumers for participation in the economy? Is it to create an informed electorate? Is it to indoctrinate children into a way of thinking that will not disrupt established order? Is it to create life-long learners? Is it to prepare students for their past or is it to prepare students for their future? I ask this question a lot and I get a lot of different responses. High school seniors usually tell me the purpose of school is to help people get a better paying job. School teachers usually tell me it is to pass on a love of learning. Politicians usually say it is either to create an informed electorate or to prepare young people to compete in a world economy. What is it? One would be tempted to say yes to all of the above but there presents some contradiction.

      Understanding by Design, your own formula, tells us that we have to start with an essential question then develop an appropriate assessment tool, and finally craft activities that will drive students to an enduring understanding. As far as scripted formulaic teaching methods go UBD is better than most and I support every teacher having a foundational knowledge of the concept. But, it seems to me value added assessment has failed to identify the appropriate enduring understanding to the larger essential question of, “What is the purpose of school?” How can any assessment measure to test teacher effectiveness be created, how can we know what constitutes a good teacher, if we cannot agree or identify what their purpose is?

      Like

  19. This extreme pressure on teachers to add value is stupid on the stupid face of it. The key problems in education are not underdeveloped teachers, though some of it involves teachers unable to work effectively in insane situations re resources, overcrowding, administrators, policies, etc. And, of course, there will always be the occasional incompetent or insubordinate teacher, but value-added nonsense is not the best way to weed them out. In fact, it may not work at all!

    The majority of the problems reside in overall school missions, methods, resources, curricula, and administrations; lack of parental involvement or family dysfunction; blighted communities and chronic student issues that affect focus, motivation, or self-perception.

    Yes, teachers are the main masters of adding value, but the real problems are the disaster areas in students’ lives, academic and worldly. That’s when their learning stalls and skills fade. That’s what has to be fixed. And to suggest that it be fixed by making everyone a super-excellent teacher to double (or triple, if need be) the learning pace is unrealistic, unfair, and evasive. Plus, this whole value-added concept has some inherent problems, besides our inability to accurately measure it. Ever heard of learning curves? Developing humans take them in different ways in different areas. Sometimes a major block, or steep bump, is natural, yet will require serious parental intervention, like a private tutor, when school intervention would be way too little, way too late. And there are plenty more nonlinear aspects to learning in all stages of our development: plateaus, epiphanies, synergies, stases, etc.

    These deformers really undo themselves. This is very convincing of their lack of education.

    Like

  20. Some classes may have students who have consistently advanced by 1.5 grade levels per year while another class in the same school has students who are improving by .5 grade levels per year. Teacher performance cannot be reliably measured without taking this into consideration.

    Linking teacher pay and job retention to scores of tests that they administer is like allowing law school students to grade their own bar exams.

    Like

  21. Grant,

    I have great respect for your work in the field, but as someone familiar with schools and research, can you please explain to me how someone using VAM can separate out all of the effects on students and isolate the effect of instruction using state tests? Every example I’ve seen recognizes the importance of having multiple years worth of data, but are the “experimental conditions” similar enough year after year? Are the teachers working in the same conditions, with the same supports? This type of analysis is so cold and abstract that it frustrates those of us close to the work. One of my students committed suicide one year. Do you think that matters? How about you, John? Is there a space in the data spreadsheet for that? Another year, my son was born and I missed a few weeks of school. Is that allowable? Is there a space in the spreadsheet for that? Do you think that three weeks of substitute teaching might affect the class? One of my colleagues just went through a fight with cancer and is doing great. Is there a mark on the spreadsheet for “Cancer survivor” and a way to keep that year of data out of “her” test results, when much of the year was with a long-term substitute whose name will NOT appear on the students’ records? The folks who push this stuff down our throats at the school and classroom level NEVER seem to have answers to these questions. You talk about data and statistical models and I’m working with life and death here. I remain optimistic that no sane principal would fail to notice and account for these factors, but that’s not what we’re talking about here. The L.A. Times is absolutely wrong on this one, and their apologists have no satisfactory answers for the realities that complicate their data analysis.

    Like

  22. Sadly, Wiggins goes further than even the Times or Merrow in debasing teachers. He equates public school teachers teaching low-scoring kids in high-poverty schools with with millionaire, weak-hitting baseball player/entertainers. Even a pro ballplayer isn’t judged on batting ave. alone. He’s part of a team. Can he field? Hit for power? Steal a base? In a slump? He gets the best coaching money can buy. Wiggins’ problem is, he loves standardized tests and claims more from them than do the test makers themselves. Test scores are not batting averages and cannot–should not be solely equated with the skill of an individual teacher. Wiggins should know that.

    Like

  23. John, echoing what David B. Cohen says, if you don’t endorse simple bubble tests or simplistic value added measures, and if you don’t endorse using only VA measures to evaluate teachers, I’m surprised that you liked the Times article.
    What bothered me about it was that they put in caveats about VA methodology and mentioned the National Academy of Science position, and but did so *in passing* and then wrote about teacher A and teacher B as though it were quite clear that A was great and B a loser. I think the average reader walked away from the piece without any appreciation for the controversy of VA models and their limitations.
    As for the benefit of a piece like this for bringing issues to the fore, I recognize that reasonable people can differ, but I don’t feel good about it. It reminds me, actually, of the arguments around “The Bell Curve” fifteen years ago. The authors made that argument and pleaded that this was a free speech issue. . .in my view if you say something very controversial, the backing of which is suspect, I don’t give you a lot of points for “starting a conversation.” What you’ve done is leave a lot of people with bad information.

    Like

  24. Several, including you, appear to be writing off the statistical concerns regarding VA assessment as tangential – a mere distraction that will work itself out – and accepting that VA estimates of teacher effectiveness are sufficiently accurate.

    One comparison made on twitter today was that no-one is up in arms about the fed gov’t using chemical testing and “data” to identify whether gulf seafood is contaminated. Somehow, this strange comparison was intended to justify the use of VA measures for determining which teachers are contaminating our childrens’ learning.

    I would argue that the seafood industry would be rather upset if the likelihood of a false positive for chemical contaminants was over 10% or the overall error rate for determining whether there were or weren’t contaminants was 25% to 35% (error rates for classification in VA measures on teachers).

    Imagine the fallout to the seafood industry from numerous false positives (saying there is contamination when there is not), or the distrust that would emerge if many became ill as a result of false negatives?

    These technical concerns are not tangential. They are central to this discussion. One cannot, in this case, simply assume that the measures are sufficiently accurate for this purpose, even if we go so far as to accept that test scores and achievement gains are the central objective of teaching and conceptually the best measure of teaching quality.

    Like

  25. It is deeply upsetting to me that whenever I read something about education that POVERTY is consistently ignored (deliberate?). While NCLB & testing was developed to close the achievement gap there is a larger problem not being addressed. Children who live in poverty are already at a disadvantage. They do not have the same tools, conversations, or experiences as this who are not poor. Over time this compounds. The real question for me is…what are we doing to address poverty? And in addressing poverty how are educators providing sufficient services & support so that the gap is closing. In most cases we are not. Funds are constantly being cut, particularly in areas that are less affluent. This then keeps poor students even more behind. Let me be clear…parents, no matter their economic status want their children to succeed. I do not blame them. I blame ALL of us. As a society we have failed all of out children if we cannot ensure that they ALL have the education they need & deserve.

    Like

  26. This whole issue revolves around your question “But is it wrong to speak the truth?” Adolf Hitler and his followers thought they knew the truth. The radical republicans have convinced 18% of Americans that President Obama is a closet Moslem and was not born in this country. So do these comparative test scores “speak the truth” about whether a teacher is good or bad? You think these one-day snapshots are the one and only indicator of teacher effectiveness.

    You mention the Mr. Smith and Mr. Aqular scenario. Why did Mr. Smiths test scores go down from one test to another while the other teacher’s went up? Let us count the million ways… Perhaps Mr. Aquilar cheated, Maybe Mr. Smith’s students came from a neighborhood that had a drive by shooting the previous night and his students were traumatized by the death of a peer the day of the test, maybe there was record heat the day Mr. Smith’s students took the test and they were dehydrated and lacked the air conditioning Mr. Aquilar’s students did. I think you get the point. The scores that the L.A. Times are publishing are one-one millionth of the “truth” about teacher effectiveness. Maybe Mr. Smith saved a student from committing suicide, found a job for another student, worked with an obese student to reduce their weight. Where does the L.A. Times report on this?

    This kind of witch hunt will accomplish the opposite of what we really want. Good teachers will leave the profession because of this testing and we will be left with test prep hacks that do no good for students or this country.

    Like

  27. Can I throw something else into the mix? I am an avid education reader and have read each and every education article published by the L.A. Times in the last 10 years. A few years ago I noticed the trend in the Times towards debasing and demeaning public schools, their teachers and their unions. It became so noticeable that the Times even published an editorial admitting their rancor against unions (a union rogue burned down their building in the 20th century and they never got over it.)

    Concurrently, non-stop positive press coverage of charter schools, public school choice, and privatization of public schools came into play. That the Times, at the very highest echelons, would support the public shaming of teachers is not surprising and is a continuing part of the the trend to demean teachers and devalue the teaching profession. Nothing could be more clear after the way that accomplished teacher Karen Caruso was portrayed in the article.

    What no one has mentioned is that 40,000 students in Los Angeles attend charter schools and their teachers’ names and scores were NOT included in the expose. They will not be included in the database either. According to the reporters in a chat, charter schools have not shared those scores with the district therefore they could not be included. So while public school teachers close their summer with a blindside on their life’s work, charter schools can continue to sit in the cat-bird seat for the time being. Additionally, lost in the uproar was the fact that the most recent test scores released this week by the CA Department of Education show a clear increase in the test scores of LAUSD students and not so much for students in charters. Did this get any press coverage? No.

    My point? L.A.Times education articles have a slant and an agenda that supports the privatization of public schools. They read more like propaganda, like mouthpieces for the edu-preneurs and philanthro-capitalists, and this article was no different. I am unable to consider L.A. Times education reporting as factual, knowledgeable or meaningful, and that is a sad state of affairs in a democratic society.

    Like

  28. It’s beyond the scope of newspaper journalists to set their own standards for evaluating a profession, and then publicly name the individuals who fail to meet those standards. It’s outside the role of journalists, outside their area of expertise, and outside the bounds of impartiality, fairness and objectivity. The concept is a spectacular collapse of journalistic standards, and it’s surprising that any thoughtful journalist would support it.

    Like

  29. Leaving aside the ad hominem stuff, I am disappointed by the attacks, largely because no one is offering alternative ways to evaluate teacher performance. I wish my critics would go back and reread what I wrote, because I took pains to call out administrators who have not acted responsibly. No one seems to deny that our schools tolerate mediocre (and worse) teachers. Those days are over, and I think teachers and their leadership ought to be at the table as the ways of evaluating teachers are being decided. Proactive, not reactive.

    Like

  30. Does it serve young people for the general population to know in which classrooms student achievement is increasing, as measured by standardized tests? Yes, I think it does.

    Are standardized tests sufficient to determine the quality of what is happening in a classroom. No, I think we all agree on that.

    To me, far more offensive by far than the LA Times story is that millions of tax dollars have been paid to teachers who have been “certified” by a National Board that, according to its founding director, has no interest in determining whether achievement is increasing in their classrooms (using ANY measure or measures), whether students regard the teachers as fair or unbiased, or what parents think of these teachers. Having know some teachers who are nationally board certified, I found some of them to be some of the cruelest, most unpleasant and racist people I’ve met in years. But they were really good at putting together portfolios.

    Those dollars could have been used for things that would have made much more difference for youngsters from low income or limited English speaking families.

    I hope that LA Unified and the teachers union DO proceed to develop a more sophisticated way of measuring what is happening in classrooms, and I hope that the LA Times covers that too.

    Like

  31. Many distinguished scholars with more experience than I have commented on this article. I find many of the arguments persuasive, but continue to ask myself this one question: If we hold students accountable based on how they perform against a set of permformance standards and not each other, then why are we comparing teachers to each other rather than to a set of agreed upon standards? When we score a student’s work with a rubric, we look at how the student performed on the standards being assessed. If a student earns an A, but then the next student comes along with an even better project that exeeds the standards, does that devalue the previous student’s grade? Wouldn’t accountability be more meaningful if everyone had a voice in creating the rubric, if the expectations were based on the reality of the individual circumstance, and if the teacher were evaluated according to how well they met those agreed upon targets?

    Like

  32. I am more than disappointed by Joe Nathan’s last comment. From a discussion on aspects of Value-Added Assessment and how it should be used he decides he has to take a pot shot at National Board Certification. Are you then going to be consistent and criticize the Gates Foundation for funding an effort to see if the way NBPTS evaluates the videos submitted by candidates for National Board Certification can be used as a basis for teacher evaluation outside of the National Board process.

    John, you know my concerns about this piece, and I will not rehearse them here. Let it suffice to say that psychometricians are quite concerned with the current status of value-added methodologies and as a whole are strongly opposed to using them as a primary means of evaluating teacher performance. Somehow the failure of many advocating their use is reminiscent of their unwillingness to pay attention to what NCME, AERA and APA have said for years, that tests designed to allow drawing of valid inferences for one purpose (eg, what a student knows) cannot be relied upon to draw valid inferences for other purposes (eg, how effective a teacher is) – the statistical manipulations applied to data cannot really make it any more accurate that the underlying accuracy of how the data was obtained, and as one increases the number of factors for which one tries to control the accuracy of what is left goes down.

    Grant, reading your comment I am having trouble relating that to work you have done on authentic assessment. If you admit the problems with the kinds of tests we currently have, how is using something inaccurate an appropriate way of addressing a separate issue, which is that we have teachers who do not belong in the classroom? Don’t you undercut what you are trying to achieve by using data that does not necessarily hold up?

    Let me add one additional requirement. NCLB required highly qualified teachers. Someone who was fully certified under state law was therefore considered more important to keep than someone in a transitional program still achieving full certification, but a better teacher. If we want to address the problems of teachers, focusing on the end results, whether snapshots of student performance or statistical analyses entirely misses the key points.
    1. We need to do a better job of recruiting people into the profession
    2. We need to do a far better job of training and preparing people for the task of teaching
    3. Administrators need to exercise more care in hiring
    4. New hires need to be supported and mentored
    5. Administrators need to document where there are problems, and act to remove those who are not working out, despite being supported and counseled, before they are giving tenure or continuing contract status.

    I view was the LA Times did as exceedingly irresponsible. I have refrained from blogging directly on it, but am finding myself writing so much in response to what others have offered that perhaps I will, if I can find the time in the forthcoming first week of school, to address it.

    Let’s be clear. We have problems with a portion of our teaching staff. That has been true for quite some time. Let’s also be clear, some are using the existence of that problem not to address it, but to use it as an excuse for bashing teachers unions, undercutting public schools, privatizing education. On the one hand they criticize teachers for not addressing the problems among our brethren while at the same time excluding our voices from the discussions on policy – by and large we have NO SAY in who gets hired, in how teachers are prepared for certification, in how they are supervised. We sure as hell have no authority to fire teachers.

    If you want to address the problems with teaching, perhaps including the voices of recognizably good teachers might be a part of it? And Joe, you might be surprised, but increasingly those who are recognized are National Board Certified. Look at state teachers of the year, or locally where I am, the Washington Post Agnes Meyer Outstanding Teacher honorees (disclosure, I am one for 2010). The proportion of those who are NBCTs far exceeds the percentage of the total teaching population that is NBCT. Now there’s a data point for you to consider before you bash the national board process, and before you rely upon a process as flawed as that used by the LA Times to trumpet whom it thinks are good teachers and who are not. In my own school, many of those who are very good teachers or better who are not close to retirement are either already National Board certified or undergoing the process. Why? Because it is the best damn professional development most of us have ever seen. We find going through the process makes us better teachers, even if our peers, administrators, parents and students already think we are good teachers.

    Enough from me. Done unloading.

    It is first and foremost about the students. And that means it had better be about something far more than test scores, even value-added scores, even if they were meaningful information, which for the most part they are not.

    Like

  33. John,

    Interesting discussion. But it seems like the L.A. Times story has opened the teacher-bashing door a crack wider. Talk about “ad hominem.”

    Here’s Joe Nathan: “Having know some teachers who are nationally board certified, I found some of them to be some of the cruelest, most unpleasant and racist people I’ve met in years. But they were really good at putting together portfolios.”

    How about Grant Wiggins: “Worse, most teachers are afraid of student and parent feedback and rarely solicit it.”

    Pretty disgusting stuff.

    Like

  34. Merrow says, “everyone in any school can tell you who the really good teachers are in the building. Whether they will tell you is another story, perhaps, but everyone knows who’s good and who’s bad.”

    Come to think of it, that’s pretty amazing when you realize that the debate about what’s good teaching goes on unresolved nationwide after a century. Merrow says the problem is that “everyone” knows but the “system” isn’t dong anything about this common knowledge by getting rid of bad teachers. How would you like to teach in a place where your job and pay depended on the “everyone knows” approach to evaluation?

    I’ve taught in schools where some people, including by fellow teachers and parents, thought I was a bad teacher because of my ethnicity. Others thought I was great. In fact, I was probably somewhere in between, since I had only been teaching a few years. Where and when has this school-wide consensus emerged, Mr. Merrow? And why is it so clear to you and no one else?

    Teachers should be evaluated by peer professionals as part of a trusting relationship among them and their principals. If they are found lacking, or lacking is certain areas (as we all are) they should get lots of support. All new teachers and many old ones fall into this category. Statistics alone won’t suffice. Neither will prejudices like those expressed by some of the commentators above.

    Like

  35. Shocking, awful, embarrassing–especially since I have long admired you both–Grant and, John.

    I often thought Grant’s thinking cool/cold/logic without the common human touch, but I also respected the insights that flowed from his logic. I just can’t believe you and he wrote that junk, John. What do you think it does to kids, families, human beings…even if the test evaluations were a good measure. Nobody in the field of testing would argue for it–as you surely know. Even when I fired people for far better reasons, I did it in ways that would cause the least hurt possible. Teachers who are unsuccessful are not criminals, or bad people, or deserving of being mistreated. It’s a blow against our common humanity–surely the most precious thing we have to pass on to our children. By our way of treating each other shall we be known. (Do you imagine the possibility of this being done to one of your own offspring??? In any field?)

    I presume you’d like us also to go back to the days when the kids scores are publicly posted too.
    Maybe we can add their families–to spread the “shame” as widely as we can.

    Deborah Meier

    Like

  36. John,
    The naming of names based on dubious measures is truly disgraceful.
    I am disappointed and shocked to see you endorsing this approach.
    Why humiiate people in public when most psychometricians and accountability experts say that VAA is not ready for prime time? Shall we put scarlet letters around the necks of teachers whose kids don’t get higher test scores? Continue on this path, and we will have more teaching to lousy tests, more narrowing of the curriculum, and more cheating. Not good education. Not a way to improve education. Just mean-spirited and pointless.

    Diane Ravitch

    Like

  37. John – I realize that your original post dealt with the administrators who should be accountable for results, and that was a point worth making. However, as I see it, there are two main issues developing in this comment thread. One concerns teacher evaluation practices, and one concerns the lack of journalistic integrity at the L.A. Times, and your defense of their reporting.

    It is not incumbent upon the critics of the L.A. Times to come up with a better method of evaluation. The criticisms of VAM and of the Times will both stand on their own merits. The reporters’ conclusions are not supported by their data unless you’re willing to defy the standards of the profession and of research. Would you defend the Times if they selected a single measure of therapeutic effectiveness and made cause/effect pronouncements about a drug, in defiance of any standards established by the medical and scientific communities? It is actually an ad hominem attack on your part, to ignore the substance of the criticism (repeatedly) and focus on the critics, no? I do agree with you that from a political standpoint or a public relations standpoint, it is helpful to offer an alternative, but the validity of one’s argument about one idea is not dependent on the presentation of an alternative idea.

    Here in California, a group of teachers, including me, joined together to form Accomplished California Teachers, and we have published a set of policy recommendations* to improve teacher evaluation, by the way. (And I let you know about it when it was originally published back in June). So, if you do insist on the idea that we must offer an alternative, there it is. Does that increase the likelihood that you will reflect on your defense of the L.A. Times for a moment, and maybe even admit to any rethinking of your position? Is there any expert or scholar in the field of education who has not weighed in on this subject whose opinion might sway your thinking? What would it take?

    * [ http://accomplishedcaliforniateachers.wordpress.com/act-publications/ ]

    Like

  38. John,

    I’ll just add one note. From the following comment you made, I am not sure you read the entire article carefully: “I firmly believe that just about everyone in any school can tell you who the really good teachers are in the building.” Do you remember the passage where the reporters specifically argued that the “really good teachers” whom “everyone in any school” can identify are in fact not so at all?

    The reporters affirmed the consequent (assuming that the test scores they wanted to be important in fact trumped professional judgment and observation), and it looks like you approve their project while stating explicitly that they are wrong.

    Can anyone tell me where I am misreading all this?

    Like

    • Good observation, Sherman! I missed that contradiction. It is true that parents and other teachers get a sense of which teachers are “good” in a school. Interesting if that desire to have your kid in the class with the engaging teacher who makes them work and gives them work suited to their abilities and interests may not be the teacher whose tests scores are highest.

      I can tell you that the more emphasis is being put on the scores the less accurate they seem to be in telling me, as a ninth grade teacher, anything about the actual performance of a student. I had a student two years ago who had scored 100% on conventions (grammar) who couldn’t write a sentence without at least three egregious mistakes, and one this year who was supposedly proficient in reading but who couldn’t read at even a seventh grade level.

      What gives?

      Like

  39. All the research shows that parent involvement plays a major role in student success, John.

    How about publishing data on each parent’s value-added? In other words post parent’s picture, address and phone number alongside their kid’s test score yearly gains or losses along with a description of activities, like turning of the TV, helping with homework, etc… that they provided or failed to provide. Absentee fathers would be featured with major head shots. After all, bad parenting is more costly to the taxpayer than poor teaching. Plus you need to be credentialed to teach. L.A. Times board would love it. Think of the boost to circulation. Everyone would by the paper to see if anyone in their family was being humiliated today.

    Like

  40. It was Diane Ravitch who recently said in a webinar, basically, that “teacher effectiveness” is a red herring and a distraction from real educational concerns. These concerns for most public school teachers include the inequities faced by children of poverty and color; the fracturing and demeaning of public school systems by the charter school and “public choice” movement supported by those with vast sums of money to push their own educational agendas, research-based or not; the constant bashing of teachers and teacher unions; and the unending parade of simplistic solutions (e.g., publicizing standardized test scores, “choice”, etc.) to the complex societal ills that are not being addressed. The Los Angeles Unified School District is trying to get teacher evaluation and teacher effectiveness issues right. It is outrageous to me that three reporters from the Times feel they have the authority and the ability to publicly label teachers as either effective or ineffective based on one test, when the LAUSD hasn’t yet come to firm consensus on what effectiveness even is after months of discussions. And it is irresponsible and outrageous to me that Grant Wiggins asserts “…most teachers are afraid of student and parent feedback and rarely solicit it”. Grant, you ignore your own background in good research when you make that unproven allegation so public. Even my sixth graders could tell you on our state standardized tests that your assertion is unsupported by evidence and that the term “most” likely indicates a wrong answer.

    Like

  41. Richard,

    After your honest, inspiring words nothing else needs to be said.
    I’m so happy the summer’s almost over, so we teachers can get away from all brew-ha-ha, button up our shirts, and teach our students as best we can.

    Like

  42. As an elementary educator for 21 years, I am simply disheartened and horrified at what is happening in public education since NCLB and the increasing focus that is being put on standardized test scores. the current focus on teacher’s test scores is like a witch hunt! The students that I teach are “tested to death.” We have so much data we can hardly address the area of concern on one exam before another exam is due. Meanwhile….

    I teach children who are second language learners and who, for the most part, live in poverty. They do not have many culturally enriching experiences and some have never been to the beach though they live a few miles from the Pacific. Many students come to school over tired due to sharing a bedroom with multiple siblings and having the television on late into the night.

    We maintain high expectations at our school and work so very hard to bring the students up to grade level standards. It’s quite a task to make up for early years at home when these children were not read to or when education is not a major value in the home.

    Yes, there are ineffective teachers and everyone knows who they are. so, why are they given an evaluation by the administrator which requires improvement. By the way. that will not show up solely in test scores. It will show up in classroom environment, student motivation and respect for their teacher, the rapport between students and teacher, and the level of student engagement with what is being taught.

    However, with the amount of testing and test preparation that is being done, it is hard for a child to be a child anymore. Art is nearly non-existent as we push to bring these kids up a band on “THE TEST.” There are no plays to act out for peers. There is no painting unless you dare to get off of your pacing guide and risk being a bit behind when that next test on the way to the big test comes along. School is becoming a factory. Good and devoted teachers as well s the students they teach are undervalued in today’s system.

    I could weep. Instead, I get up and go to work and look in students’ eyes, find out if anyone is hungry and get on with the business of teaching in between listening to stories of wife beating, drunkenness, lack of sleep, lack of money, jail and divorce. Just another day of getting ready for the test.

    Like

  43. Face it. They did this story because it was mostly easy to do … the data was already there, there are always people available to interpret the data and draw conclusions, and the conclusions drawn had shock value, made great headlines, and drew national and probably international attention. Most people don’t understand test results, issues schools deal with, the difficulties with evaluating “good teaching,” and more. Our journalists and press have abandoned us – they do what is easy and shocking because it is cost effective and won’t ruffle too many feathers. Let’s do some deep stories on the other issues that hold back students, teachers, administrators, schools and education in general. What are the other big issues? How about all these large textbook companies that continually promise huge test score gains if their series is purchased for millions of dollars. Are they part of the problem? Is societies reluctance to change a model of school that is over a century old because change is hard part of the problem?

    We have decided that education issues are easy when they are not. So instead of investing in finding the best answers we can, we rely on assessments that everyone agrees are poor at best (but they’re what we have), and because dealing with issues like poverty, indifference, language, teacher effectiveness and more are hard, very hard, we develop an education plan that attempts to step around them. President Kennedy, when he spoke about going to the Moon, talked about doing so because it was hard. Look what breakthroughs came from the work and research that went into that endeavor – in computer science, science in general, nutrition, health, materials and so much more. We have the money … those billions of dollars that are earmarked for a very flawed and narrow RTTT.

    Let’s re-purpose some of that money to do the R&D necessary to REALLY make a difference. Let’s fund many approaches instead of mainly KIPP clones when no one associated with KIPP will send their own kids to the schools they advocate. What breakthroughs in pedagogy, health, psychology, family/parent involvement, art, math, reading, science and so much more will we discover? They would be exciting times that just might rival the space race. What implications might that have for the rest of the world too?

    Do we need to find and deal with the poor teachers in our ranks? Yes! Do we need to make our schools the best they can be for now and our future instead of just making another tweak of a 100 year old + model for schools that we hang onto because it is what we know and change is hard? Hell yes! So let’s do that instead of wasting time on bandaid, easy approaches.

    Oh, and you really need to re-think your support of what the Times did, and exactly how they did it.

    Like

  44. There seems to be some inconsistency here. You admit that standardized test scores alone are not a valid measure of effectiveness, yet you applaud the release of pictures and names of teachers judged ineffective on the basis of those scores alone. You say this is okay because everyone knows who the good and bad teachers are anyway. Yet one of the teachers named as ineffective in this piece was considered a good teacher by her administrator. She’s even National Board Certified.

    Further, you say administrators should be held accountable for not helping teachers improve, because those administrators have access to the data. But this piece is not talking about ineffective administrators, it’s nailing teachers.

    Student standardized test scores are simply not a valid measure of teacher performance. Even if they were, whisperings in the school hallway about a teacher’s performance is one thing. Publishing names and pictures in a nationally-read newspaper in an article that is sure to be relinked around the world is another thing altogether. These teachers are hard-working people, and they deserve a little more dignity than this.

    Like

  45. How many “bad” kids are there that are forced in the system, some from high ses and some low, that have many talents and abilities that don’t fit “school” and are wasted? How many “bad” students thrive after they leave school? Not just the Steve Jobs, but also all the anonymous ones? The same goes for teachers. We view bad teachers and say “We must put more pressure on them or counsel them. (so they will fit into our dysfunctional system better)” It is far more difficult to adjust the system, on the school, local, state, or federal level, to accommodate and support the different talents, skills and abilities of teachers AND students that don’t fit “school.” These are the tragically rare and special leaders who are able to do this, despite the system.

    Like

  46. I think people are losing sight of a key issue here: administrators have known for years which teachers are not able to teach effectively–and have done little or nothing about it. Would I have been happier if the LA Times had interviewed the principals to determine why they had not acted? Yes. Would it have been OK if the Times had just used first names only? Sure, and perhaps that would have put the focus on the core issue, the system’s tolerance for mediocrity.

    But the old trade union mentality–teachers should be paid (and judged) based on years in the classroom and graduate credits–is simply indefensible. And most of the fine teachers I know do not even attempt to defend it.

    Unfortunately, that mentality–avoid connecting the dots between the adult and the students in his/her charge –is, unfortunately, alive and well. Below is an excerpt from my interview earlier this year with the president of the National Education Association. I wanted to know if student outcomes were a legitimate way to measure teacher effectiveness. In a word, NO, he said, because what matters most is how the teacher conducts himself in the class, not whether the students master the material.

    MERROW: Can you judge a teacher in part based on how his or her students perform?
    DENNIS VAN ROEKEL: When I think of practice, as a … for twenty-three years, as a high school math teacher, how I use test scores to impact my practice and that’s what I should have been judged on, I believe. If … if, for example, in a class, you know, you never know which part of a … a geometry lesson a class isn’t going to get. If based on my assessments, uh … a quiz or whatever, and they didn’t get my unit on slope, then what they should be watching for is what did I do as a result once I realized that they didn’t know it? Did I adjust my teaching? Did I find a new way of doing it? That’s what I should be judged on.
    MERROW And if you didn’t?
    DENNIS VAN ROEKEL… on my professional practice.
    MERROW And if you didn’t?
    DENNIS VAN ROEKEL If I didn’t, I think that’s part of the evaluation system.
    MERROW So?
    DENNIS VAN ROEKEL So it’s the practice, not the test score.
    MERROW But the test scores are the measure of the practice.
    DENNIS VAN ROEKEL I don’t believe that.

    (You can hear much more of this interview on a podcast on http://www.learningmatters.tv)

    So If I take my car in to be washed and waxed and it comes out dirty, I have no recourse as long as the employee used all the right motions? Or if the swimming coach followed the book on teaching swimming to the letter, his job is safe even if a few kids drowned?

    Teachers are going to be evaluated on student performance, like it or not. It behooves us all to work for better and more comprehensive measures, as Joe Nathan said. David Cohen’s reference to the California model is worth a look as well. It accepts outcomes as one measure and argues for more depth.

    I am not endorsing the ‘everyone knows’ model. I know from my own experience that many, perhaps most schools have teachers who are simply going through the motions. That no one calls them out is a disgrace. One of my failures as a reporter is the fact that I have not done more reporting on that issue.

    As I argue in Below C Level, the public education system’s high tolerance for mediocrity is the heart of our education system’s problem. It’s not unions, or school boards or bubble tests or anything like that. This nation cannot survive if we continue to be unconcerned about being below C level.

    Like

  47. Using standardized test scores as a means of judging good teaching and real learning is a morally bankrupt and intellectually indefensible position to maintain. John Merrow and Grant Wiggins should know better.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s