These five test questions may explain why American students score lower than their counterparts in most other advanced nations. The first is a sample problem offered by the University of Wisconsin/Oshkosh {{1}} to high school math teachers. It was designed with the stated goal of ‘Closing the Math Achievement Gap’:
Jack shot a deer that weighted (sic) 321 pounds. Tom shot a deer that weighed 289 pounds. How much more did Jack’s deer weigh then (sic) Tom’s deer?
Basic subtraction in high school? The second comes from TeacherVision, part of Pearson, the giant testing company {{2}} :
Linda is paddling upstream in a canoe. She can travel 2 miles upstream in 45 minutes. After this strenuous exercise she must rest for 15 minutes. While she is resting, the canoe floats downstream ½ mile. How long will it take Linda to travel 8 miles upstream in this manner?
While the second problem does not contain language errors, its premise is questionable. Will some students be distracted by Linda’s cluelessness? Won’t they ask themselves how long it will take her to figure out that she should grab hold of a branch while she’s resting in order to keep from floating back down the river? What’s the not-so-subtle subtext? That girls don’t belong in canoes? That girls are dumb?
And I found this on a high school math test in Oregon:
There are 6 snakes in a certain valley. The population doubles every year. In how many years will there be 96 snakes?
a. 2
b. 3
c. 4
d. 8
Remember that these are math problems for high school students! They require simple numeracy at most, and “Snakes” can be solved by counting on one’s fingers.
Next is an example of what lies ahead for 8th graders–not high school students–under the new Common Core National Standards, which are supposed to introduce much needed ‘rigor’ to the curriculum. This question (without illustrations!) is from New York State’s sample tests.
Triangle ABC was rotated 90° clockwise. Then it underwent a dilation centered at the origin with a scale factor of 4. Triangle A’B’C’ is the resulting image. What parts of A’B’C’ are congruent to the corresponding parts of the original triangle? Explain your reasoning.
Did you go ‘Huh?’ I did.
The fifth and final question was given recently to 15-year-olds around the world on a test known as PISA (for Programme in International Student Assessment):
Mount Fuji is a famous dormant volcano in Japan. The Gotemba walking trail up Mount Fuji is about 9 kilometres (km) long. Walkers need to return from the 18 km walk by 8 pm.
Toshi estimates that he can walk up the mountain at 1.5 kilometres per hour on average, and down at twice that speed. These speeds take into account meal breaks and rest times.
Using Toshi’s estimated speeds, what is the latest time he can begin his walk so that he can return by 8 pm?
For simplicity, let’s call these problems ‘Deer/Canoe/Snakes,’ ‘Triangles’ and ‘Fuji.’ Those high school problems are far too easy. With enough practice, just about anyone can solve undemanding problems like that and, consequently, feel confident of their ability.
Note that ‘Fuji’ is not a multiple-choice question. To get the correct answer to this engaging question, students had to perform a number of calculations. The correct answer (11 AM) was provided by 55% of the Shanghai 15-year-olds but just 9% of the US students.
Speaking of confidence, the PISA results reveal that American kids score highest in ‘confidence in mathematical ability’ despite underperforming their peers in most other countries. Is their misplaced confidence the result of problems like ‘Snakes’ and others of that ilk?
School is supposed to be preparation for life, but spending time on problems like ‘Deer/Canoe/Snakes’ is like trying to become an excellent basketball player by shooting free throws all day long. To be good at basketball, players must work on all aspects of the game: jump shots, dribbling, throwing chest and bounce passes, positioning for rebounds, running the pick-and-roll and—occasionally–practicing free throws.
Come to think of it, basketball and life are similar. Both are about rhythm and motion, teamwork and individual play, offense and defense. Like life, it can slow down or become frenetic. Basketball requires thinking fast, shifting roles and having your teammates’ backs. Successful players know when to shoot and when to pass. As in life, failure is part of the game. Even the greatest players miss over half of their shots, and some (Michael Jordan!) are cut from their high school teams. And life doesn’t give us many free throw opportunities.
But if school is supposed to be preparation for life, why are American high school students being asked to count on their fingers? That mind-numbing and trivial work is the educational equivalent of shooting free throws.
Now to ‘Triangles,’ which represents education’s brave new world of the Common Core, adopted by 45 states and the District of Columbia. In this new approach, students will be exposed to higher and more ‘rigorous’ standards. The hope is that the curriculum, locally developed to reflect the standards, will challenge and engage students. I suggest you read ‘Triangles’ aloud.
Triangle ABC was rotated 90° clockwise. Then it underwent a dilation centered at the origin with a scale factor of 4. Triangle A’B’C’ is the resulting image. What parts of A’B’C’ are congruent to the corresponding parts of the original triangle?
Are you feeling ‘engaged’? Imagine how 8th graders might feel. If ‘Deer/Canoe/Snakes’ are the educational equivalent of practicing free throws, then solving problems like ‘Triangles’ is akin to spending basketball practice taking trick shots like hook shots from midcourt—another way not to become good at the sport. I worry that questions like ‘Triangles’ will impede the understanding and appreciation of math for the 99% who are not destined to become mathematicians.
If our schools persist with boring, undemanding curricula, our kids will be stuck at the free throw line, practicing something they will rarely be called upon to do in real life. If, however, in the name of the Common Core’s ‘rigor’ we give our kids lifeless questions like ‘Triangles,’ schools may end up turning off the very kids they are trying to reach.
—-
[[1]]1. Wisconsin source: http://www.uwosh.edu/coehs/cmagproject/many_word/documents/1_Dear_Hunting_Problems.pdf The web page notes that ‘slightly more difficult problems are coded with a shamrock/diamond.’ Here’s one of those: “In the first half of a recent game, the Packers scored 14 points on touchdowns and 9 points on field goals. In the second half, they continued scoring, and they ended the game with 43 points. How many points did they score in the second half?” [[1]]
[[2]]Pearson source: https://www.teachervision.com/tv/printables/botr/botr_140_3-3.pdf [[2]]
The Merrow Report 
Everything you say is spot on accurate. You correctly write: “School is supposed to be preparation for life, but spending time on problems like ‘Deer/Canoe/Snakes’ is like trying to become an excellent basketball player by shooting free throws all day long. To be good at basketball, players must work on all aspects of the game: jump shots, dribbling, throwing chest and bounce passes, positioning for rebounds, running the pick-and-roll and—occasionally–practicing free throws.”
Please answer this question, though:
How can school ever be anything other than shooting free throws (at best) if students are never integrated and participating in the outside world, doing, experiencing, living, and developing skills?
LikeLike
Unless your goal is to teach math, I’m not sure what value the “triangle” question is – and I love math (most really not used in my engineering / college teaching careers)!!! Other than early in my teaching career when I thought solving DE / PDE equations was important to engineering careers (it’s probably not), I only solved ONE DE in my career of 40 years. But the triangle question is simply testing math vocabulary (not sure I ever learned dilitation …). I agree with the author!
Back to calculus thoughts. Lest you think I believe all my education was wasted, I don’t! I only wish my students understood and could used the basics of calculus: what slope and integration represent, what limits represent, … Believe me, many struggle with the book definition; most couldn’t apply this “knowledge” if their lives depended upon it!
LikeLike
“How long will it take Linda to travel upstream in this manner?” How far upstream? Silly, incomplete question. Silly tests. Woe is us.
LikeLike
My bad on that one. I inadvertently left out the mileage when I retyped the problem. 8 miles. It’s in the problem now..
LikeLike
Becoming a competent basketball player requires many hours of mastering basic skills.
LikeLike
Nice blog post. I have been successful in a financial and consulting career before finding my true passion in education. In each of my careers, the math expected to be understood and applied in these examples was never needed for me to succeed. While I am providing anecdotal evidence, my reality may be more the norm than expected. Not to negate the importance of math in certain fields, but I happen to believe the economy in our future will require far more right brain intelligence. My fear is the over emphasis on high stakes testing and preparation for this testing will beat the right brain thinking out of our students.
@mitchellsensei
LikeLike
I agree! Having been an art teacher, I sadly observed that the more emphasis on tests, the more students wanted ‘one right answer’. Very sad. They were afraid and hesitant with ambiguity, choices, and the serendipity of failure – essentially much of the arts. I hope we have more brave teachers and students who find joy in ambiguity, challenge in the abstract, and inspiration in playing with ideas!
LikeLike
“Triangle ABC was rotated 90° clockwise. Then it underwent a dilation centered at the origin with a scale factor of 4. Triangle A’B’C’ is the resulting image. What parts of A’B’C’ are congruent to the corresponding parts of the original triangle?”
FYI – the angles will be the components that are congruent to the original triangle
LikeLike
Stuff to consider regarding PISA:
http://www.upworthy.com/5-myths-about-our-schools-that-fall-apart-when-you-look-closer-6
LikeLike
The second question doesn’t state the distance Linda needs/wants to travel upstream. There is no way to answer it since the distance isn’t included. The most a student could do is state how far she can travel in an hour. Since Linda isn’t too bright, it is quite possible that she would never reach her destination because she started paddling the wrong direction once she entered the river. Poor, tired Linda.
The snake question requires that students know their math facts and in Washington State they were not required to memorize their math facts when my kids were in elementary school. My older daughter never memorized her math facts and while she could figure out the snake question, it would take her a little while. My younger daughter had a teacher for 2nd and 3rd grades who was new to the state and she required students to memorize their math facts at their own pace. My younger daughter would be angry about how dumb these high school math test questions are and she would be angry about stupid Linda (she’s a feminist).
Here’s the deal, with No Child Left Behind each state had to try to get every student to pass the test so putting dumb questions on the test was to the state’s advantage. Unfortunately, some of the questions can’t be answered because they are poorly written and/or mathematically incorrect so kids get the wrong answer because there is no correct answer.
One of the test questions my oldest had on her high school math test was regarding the factors that influence the cost of car insurance. That was the portion of the test that a calculator was allowed and there was absolutely NO math required to answer the question. The question did, however, require some knowledge of how car insurance works. Do you know any 15 year olds who buy car insurance? I sent that to our state legislators on the House and Senate education committees so they could see what the students are up against. If students don’t pass the required reading, math, writing and science tests in high school, they cannot graduate. There is a lot riding on these error-riddled tests.
I think (you can call me crazy if you want) that the goal of the Common Core tests is to “prove” that our education system is “broken” so investors can profit off things they claim will “fix” our schools – charter schools, interventions, software, curricula, etc. They also want to get rid of the experienced, expensive teachers so there is more money available to educational investors. Now the test results are tied to teacher evaluations because of the Race to the Top requirements. It is to the testing company’s advantage to make sure a lot of students fail their tests (with triangle questions) because then they can sell things to improve testing scores over time.
LikeLike
As noted above, when I retyped the problem I inadvertently left out the mileage, 8 miles, that she had to travel. It’s still a dumb (sexist) problem, IMHO, but I apologize for my error.
LikeLike
Thank you for noting that the real problem is the sexism inherent in the question. My immediate thought was that it is a trick question. My answer would have been: this person is clearly too stupid to make it upstream alive.
LikeLike
A number of thoughts
1. Multiple choice tests are poor measures, even if the questions are well designe (which often they are not). First, they encourage, even require, convergent thinking, which for real world problems is not particularly useful. Second, they can often be answered by process of elimination even if one does not really “Know” the answer. Finally, they do not allow the student to explain her reasoning process, which could point out flaws in the process.
Let me give an example from the sample question more than a decade back for the High School Assessment in Government in Maryland. The question was what would happen to representation in Congress were a 51st State admitted. The answers went something like this
A. The House and Senate would remain the same size
B. The House would increase, but the Senate would remain the same size
C. The Senate would increase, but the House would remain the same size.
D. The House and Senate would increase.
The problem, as anyone of the age of someone like John and me should remember, is that both C and D are correct, but it depends when you look.
Long term, because the House is set at 435 by legislative action, the House would be at that level, while the Senate would increase to 201 (51 x 2)
But the reality is that no state may be admitted without one House member, and no House member can be taken from one state and given to another except as a result of reapportionment subsequent to a decennial census. Thus when Hawaii and Alaska were admitted in 1959, the House expanded to 437 until after the election of 1962, thus also making answer D correct. If you doubt me on that, check the total electoral votes in 1960 and you will find 537 among Kennedy, Nixon and Harry F. Byrd.
2. Too often those that value test scores are those who do well on such tests, be they AP or SAT or ACT or whatever. They did well, they are smart, therefore those tests must be good indicators. Baloney. I do very well well on such tests because (a) I read very quickly (the results of the tests are distorted if timing is a factor); (b) I know how tests work; (c) I don’t get distracted by problems such as those I described on the sample HSA questions, because I can usually figure things out in my head. On math questions, I can use process of elimination in knowing that an answer may need to be, for example, positive, even and greater than 100, thereby eliminating 3 out of 5 answers, improving my score even if I still had to guess (which I don’t – on most SAT questions I can do all required math in my head. That does NOT make me better at Math, merely better at taking an SAT math section).
3. Raising “rigor” without providing scaffolding does nothing for the students not meeting supposedly “lower” standards, it just means even more will “fail.” Of course that presumes the standards have some reality for what students need to know and do in the real world, which is a separate question.
But simply throwing more and tougher content out without provide the time and flexibility for teachers to meet the needs of students before them is to condemn ever greater percentages to being labeled failures. That the approach the underlies Common Core and its associated tests seems to be driving in the direction of structured and rigid curricula and pacing guides is antithetical to real learning.
Let me be clear on several points
1. I personally do quite well on such tests
2. As a teacher with close to 2 decades of experience (I started late, when I was almost 50, and now approach 67), in general my students do fairly well on such tests, given their previous education and their socioeconomic and family situations. But that is in part because I have been allowed flexibility to use my professional judgment to best meet their needs.
3. I am quite knowledgeable in the fields I teach (Social Studies and now some STEM topics on media, research and policy) even though my undergraduate major (from a very elite school, Haverford) was Music. That is in part because I am a life-long learner.
4. I was thoroughly trained in pedagogical techniques, required to be reflective of my practice, and given a strong foundation on issues like infusing reading and writing across the curriculum, issues of human growth and development, and the legal and moral issues of special education (which can include students who are gifted but dyslexic or have Asperger’s).
What is ridiculous in what has been happening in education policy now for more than 3 decades (A Nation at Risk was released in 1983) is that we keep doing more and more of the same, with policy being defined without the meaningful inclusion of professional educators. I am not so unique in my ability to work in the policy arena while remaining classroom based. Yet at national, state, and district levels, the voices of professional educators are rarely a part of the development of curricular standards or of standards-based tests. The process is too often political, often with the intent of devaluing public schools, teachers and teaching, and undermining both the professionalism of teachers and the unions that exist to protect their due process rights from abusive administrators and school boards.
I could write so much more in response to this post, John. You know that, as do those who have read things I have written as teacherken, and under my own name (Kenneth J. Bernstein), here, at Daily Kos, for the New York Times and the Washington Post and Education Week Teacher, in the book reviews I have done for Education Review, etc.
If we do not step back and realize the insanity of our current national approach to education, in which the Obama administration in many ways is WORSE than its immediate predecessors, even if our scores on tests like PISA were to go up, that would not be an indication that our students are any better prepared for the real world.
The mother of three students I taught in a previous school is now on the National Assessment Governing Board. A few years ago she made a big issue of our lack of civic education, and pointed to what her children had experienced in my classes about learning to think, to challenge, to engage. Unfortunately, what we are doing in education is denying ever increasing percentages of our students to have that kind of experience.
Then again, we do have a significant portion of our population and thus our political class that apparently does not want our people to think critically, which is why they are opposed to real science, why they want to censor the literature our students can read, why they want to impose their “values” on the rest of us.
Yes, there is the greed factor, the corporate interests – especially Pearson, but also hardware and software interests (such as Bill Gates) – being far more concerned about their profits than of the real well-being of the students whom they do not teach and often whose lives they cannot grasp. There is the selfishness of politicians who seek to demagogue the issue of education for their own political advantage. There are also those who seek to have an educational advantage for THEIR children, whom they would never submit to the mandates they are quite willing to impose on other people’s kids, you know, the children of families who not like them.
I do not excuse universities and colleges that see teacher education as a cash cow, whether for original certification or the coursework required for recertification. Nor do I excuse those university-based types who seek to monetize their research, regardless of the impact upon schools, teachers, and students.
The real crisis in education is that we are destroying meaningful learning, robbing children of the joy of lifelong learning, turning school into something to be endured rather than a place of excitement, exploration, discovery. In the process we are also killing democracy.
LikeLike
Excellent post, John. Each of your four U.S. examples is appalling in its own way.
But I would quibble with your comparison of the “Canoe” and the “Fuji” examples. Both are reasonable, similarly difficult word problems, and whether they are engaging depends entirely on the reader. I’d say there is an interesting added wrinkle to the “Canoe” problem, namely that Linda (we’re now on a first-name basis, she & I) paddles for the first 45 minutes and then slides back downstream, and the backsliding won’t happen during the hour in which she reaches her destination.
The real issue with the “Canoe” problem is that it, as one comment points out, fails to specify the upstream distance that Linda must paddle. This is a truly incredible omission. By comparison, the “Fuji” problem is tersely and clearly stated, even to the point of making clear that Toshi’s estimate has factored in meal and rest breaks.
There is an unfortunate perception that math is math and writing/speaking are writing/speaking, and there is no intersection between math and writing/speaking. There is little emphasis on math as a language for describing complex situations. This produces more than simply incoherent math problems; it also produces engineers and finance professionals who may be brilliant at what they do, but are incapable of communicating their important knowledge and information to others.
–Randall Bolten, author, “Painting with Numbers”
LikeLike
Randy, as noted above, I inadvertently omitted the mileage when I retyped the problem.
LikeLike
teacherken wrote:
“Multiple choice tests are poor measures, even if the questions are well designed (which often they are not). First, they encourage, even require, convergent thinking, which for real world problems is not particularly useful. Second, they can often be answered by process of elimination even if one does not really “Know” the answer. Finally, they do not allow the student to explain her reasoning process, which could point out flaws in the process.”
And a major flaw is that such tests, traditionally, are easier to cheat on. Unless one has another basis on which to grade performance (such as short answers and essays) a student could know nothing -not even able to read or write English- and pass the test by copying and memorizing patterns of dots. Once I did an experiment by giving two tests with identical information and questions on each. One test was merely multiple choice. The other was test that require written responses (short fill in the blank or short essays sometimes as short as a sentence). It should be of no surprise that the multiple choice test was easier for the students. But what was remarkable was how much MORE I the teacher learned from the second test. Some students had terrible penmanship. Many could not even spell names as simple as George Washington! (they spelled it, quite memorably, a “Chorchi Guasinto”). But the most dramatic example was the example of a student I shall call “Jose”. On the first test, the multiple choice test he scored an 88%. On the second test he left THE ENTIRE TEST BLANK and COULD NOT EVEN ANSWER ONE QUESTION (in English or Spanish; he was allowed if he couldn’t write it out in English to answer in Spanish. It should be obvious to anyone that “Jose” had very good eyesight and copied his answers from his neighbors or was given answers from neighbors (two of the students near him scored 88 or 92). When moved to a different location and when given an objective test Jose could not perform. On the common core tests (all to be computer based) tests are going to be more random and there will be no score sheets so teachers and students will be unable to fake results. Really? Who is to say who really takes an on line test? I myself was the witness to an on line constitution test which was done via three way calling to help a student meet this requirement. If a teacher’s job is on the line will not there be temptation to coach answers on computerized tests?
Junk food is food but as junk food it should not be relied upon as the basis of a good diet. Multiple choice tests are academic junk food. They may be good as a quick dip-stick for large cohorts of students but there is really no need for them to be so prominent in k-12 classrooms. As a language and history teacher I use multiple choice tests only for ONE PURPOSE and that is to practice for standardized exams. Of course, the best standardized exams -like AP exams or the California Exit exam in English have large essay portions which makes them more authentic. And they are more authentic because if one can write a critical essay properly one is showing evidence of one’s cultural literacy and educational attainment. In life the answer is never “C”.
LikeLike
“If our schools persist with boring, undemanding curricula, our kids will be stuck at the free throw line, practicing something they will rarely be called upon to do in real life. If, however, in the name of the Common Core’s ‘rigor’ we give our kids lifeless questions like ‘Triangles,’ schools may end up turning off the very kids they are trying to reach.”
This is a very good comment, John. Test taking prep can be deadly dull. I teach a class of CAHSEE (High School exit exam English). It can be tedious to review every point of grammar, spelling, vocabulary etc. Some teachers teach this class by giving prep tests over and over again. Some teachers have the students do all their writing on the computer. There is a place for both approaches but my theory is this: One can teach BEYOND the test. The best way to teach literacy is to read and the best way to learn to write is to read and write, read and write, edit and re-write. I try to make the classes as interesting as possible with accessible poetry and some action and adventure stories as well as interesting non-fiction articles. We generally read short works but once the test has been taken (and there is a long pause before it is administered again) we read an entire book. I also use films with subtitles and extensive “movie notes” to get students to participate and respond orally in a way they may not for a written work. And of course by reading dialogue and answering questions (in complete sentences) they are READING. And every film is analyzed for its plot, characters etc. just like any other story. We see the classic film “SHANE” for example and compare the film to the novel (they are quite close but George Stevens added some character elements that are not present in the book). Most students when they come to me can write a basic marginal essay. But what they lack, generally speaking, is the depth of vocabulary to have a high reading comprehension score, a lack of understanding of literary terms and devices and a very foggy understanding of grammar and the difference between academic English and popular English. My job is make the student understand that they need to improve in these areas to score well but more importantly raise their literacy WHICH IS ABSOLUTELY NECESSARY for them to achieve higher levels of training and education. A student who passes the CAHSEE, for example, with a very low grammar score (40 or 50%) might have an excellent spoken command of English and an excellent reading comprehension but his or her writing skills will be very deficient or marginal. Such a student must have remediation after high school if he or she is going to have a chance to do college level work. Standardized tests are just a dip stick to see, superficially, how students compare to one another and how they have achieved. But they are not education. They are not learning. In fact, in many ways , they are poison for the mind. Filling in ‘snake-eggs’ does not make for a joyous learner or a deep thinker. Something to think about.
LikeLike
By coincidence, the Fuji problem is also discussed in my article
” What does the International PISA Math Test Really Tell Us?”.
It is at
Click to access JPS-Winter2014-FINAL.pdf
American Association of School Administrators “Journal of Scholarship and Practice // Research and Evidence-Based Practice Advancing the Profession of Education Administration” Winter 2014/Volume 10, No. 4; the issue is all about PISA
LikeLike