Gather round, children. Grandpa Johnny wants to show you how to cheat and lie without doing anything illegal. This will come in handy, especially if you want to run charter schools in New York when you grow up.
The story I am going to tell you, kids, involves something grownups call ‘Attrition,’ which means losing things, in this case children like you. ‘Attrition’ is bad, and so all charter schools try to have a low number, like maybe 5 or 6 out of 100.
In other words, if a charter school starts the year with 100 students and ends up with 95, that would be just a 5% ‘Attrition’ rate…and a reason to celebrate.
Here’s where you have to pay attention, because, if you run charter schools, YOU get to decide when to start measuring ‘Attrition.’ You could start counting on the first day of school, or you could wait until the state requires you to tell how many kids are enrolled a month and a half later.
In other words, if your school has 100 kids when classes begin on August 24th, you have 55 more days until you have to tell the State how many kids you have.
And here’s the best part: Whatever happens in that 55-day period does not count against your attrition because you can replace whoever leaves with new kids. It’s as if whoever leaves was never there!
So, suppose you realize that some kids who have enrolled don’t seem to fit in. Maybe they’re independent-minded, or maybe they are special needs children who will be very costly and difficult for you to educate. You have 55 days to find ways to get rid of those kids, whom you can then replace with children who are more likely to get with the program.
One technique is what they call ‘out of school suspension,’ basically sending kids home for some infraction or other, however minor. Do that a few times, and the parent, weary of having to take time off from work, will decide to find another school. That kid is gone, and it’s as if he never existed. No drag on your ‘Attrition’ rate, which is what matters to you.
There’s another way to cheat, kiddies, and that comes when your school year ends in mid-June. You lopped off the beginning of the year when you decided to count from October 7th. Now you can choose to completely ignore the summer months (a good idea because quite a few students decide to leave over the summer) and simply report ‘Attrition’ as of mid-June, your last day. Your ‘Attrition’ data covers about 155 days of school, not the full 180, but there’s no way for anyone to know that, unless you tell them (and why on earth would you do that?)
If the powers-that-be really wanted to know which charter schools lost lots of kids, the State would ask for one number, how many of the kids who started your school in mid-August of one year were still enrolled the next mid-August, 365 days later.
Come to think of it, if you wanted to run a charter school (or academy) that focused on the success of each child, then you would want to know which children did not succeed, and why. Rather than worry about your ‘Attrition’ rate, you would spend your time and money encouraging success and learning from your own failures.
But, kiddies, you don’t have to do that. You can lie and cheat with numbers without breaking any laws, you can boast about your low ‘Attrition’ rate, and you can fool the Governor, some very rich people, lots of the media, and some of the public.
Even better, when you look in the mirror you won’t have to think about what happened to the children you ‘disappeared’ during that 55-day period–because it’s as if they were never there.
Pretty neat, huh….
The Merrow Report 
Gee, John, you wouldn’t be obliquely referring to Ms. Moskowitz, would you?
LikeLike
Actually, she’s not the only one who knows how to work the system; she’s the one whose tactics have been widely reported, but she’s not the only offender.
LikeLike
I can well imagine. I hope all is well in your newly configured life. Cheers!
LikeLike
Grandpa Johnny :-), these are good facts to know. Do traditional public schools follow the same accounting rules for student attrition?
How do student attrition rates at various charter schools compare to those of the comparable traditional public schools?
LikeLike
Well, first of all, you cannot trust the attrition rates of charter schools unless you know their system. I have it on good authority that KIPP counts from Day One of one year to Day One of the next, a 365-day period, because it recognizes that some kids leave in the summer and it wants to know why.
Success Academies count from the October BED day until the end of school in mid-June. Draw your own conclusions.
Traditional public schools have open records and cannot use tactics like excessive out of school suspensions (which are forbidden in lower grades without central office approval). Their attrition is often higher, particularly in poorer neighborhoods, because of unstable living conditions. Families are often forced to move, and that means changing schools.
Any urban public school with an attrition rate of below 10% is probably a damn good school, because that low rate suggests that families will move heaven and earth to keep their kids there….
LikeLike
[Well, first of all, you cannot trust the attrition rates of charter schools unless you know their system.]
My question was: Do traditional public schools follow the same accounting rules for student attrition?
John, you didn’t answer that question and talked instead about student suspensions.
My question was: How do student attrition of charters measure up against comparable traditional schools?
John, you didn’t answer the comparison and talked solely about high student attrition in low income neighborhoods.
So, all in all, when you account for the differences in calculation logic, who has a higher student attrition rate – Success Academy schools, or comparable (nearby) zone schools?
Isn’t that the only attrition-related question that matters? And what is the answer?
LikeLike
Again, it’s worse than ‘apples and oranges.’ Charter schools manipulate the data, legally, as I noted. They CAN and DO get rid of kids, but traditional public schools cannot do this. So the comparison would be meaningless. I am not trying to evade a question, just explain why the question is not the right one to ask.
LikeLike
This is so snarky and one sided I’ve lost all respect for John Merrow. “October Count Week” has long been the law and practice in my state – it’s how ALL schools report – districts and charters. Part of the problem is that transportation chaos causes many kids who intend to attend a school wind up having to change. And, because there is no centralized enrollment, some kids appear on multiple rosters. It takes a few weeks to flesh everything out.
To slam one entity because they follow the legal and accepted practice is so biased its unbelievable. There are so many unsubstantiated and prejudicial statements in this article. Mr. Merrow – I think you ought to take the money you got paid for speaking at the National Alliance for Public Charter Schools conference a few years back and donate it to a worthy cause.
LikeLike
I wasn’t paid by the National Alliance. I showed a rough cut of a film about charter schools in New Orleans.
I know the law’s provisions, but I also know that some charter schools/networks use the period of time between their official opening and the count day to weed out certain kids. You say my statements are unsubstantiated. Please look at my piece for the NewsHour and Kate Taylor’s reporting for the NY Times on this issue regarding Success Academies. The data are there.
LikeLike
[Please look at my piece for the NewsHour and Kate Taylor’s reporting for the NY Times on this issue regarding Success Academies. The data are there.]
Kate Taylor of the New York Times is, as usual, rock solid on the factual foundation behind her stories. I may disagree with the NYT on the emphasis and perspective, but the facts are tight.
John’s NewsHour piece, in addition to some problems that deserved a rebuke fro the PBS ombudsperson, has a factual problem. It inaccurately made a connection, between a higher frequency of student suspensions at SA, and a (supposedly) higher student attrition at SA, in two ways (I am doing this by memory):
1. There is no data source showing that SA has a higher student attrition, compared to nearby zoned schools, or average traditional public schools. There is factual NYC-provided data, showing that SA has in fact lower rates of student attrition.
2. The data provided to John by Success Academy shows, among other things, every student who left the school, also showing the number of suspensions that each student received. A great majority of the students who left the school in question, hadn’t received any suspensions, so the implication that most student leave because they are pushed out by the school administration through the draconian use of suspensions, was not borne out by the data. I sourced this info from one of the redacted student stats, provided by SA to John, based on the SA press release at the time.
How do I know all this? Professionally, I am a metrics developer in the Financial Industry, and also a Success Academy parent, so I regularly look at all publicly available data, in order to tell, how my child’s school is doing.
LikeLike
Thank you for reporting on attrition rates. That is the kind of “cheating” that happens when the referees — in this case the SUNY Charter Institute — intentionally look the other way. The closest analogy I can think of is the NCAA supposedly keeping big time college football and basketball programs from “cheating”. The SUNY Charter Institute’s desire to keep charter schools from ridding themselves of students who don’t make their charter schools look successful is about as strong as the NCAA’s desire to keep big time football programs on the straight and narrow. In other words — their interest is only as strong as bad publicity forces their hand. And their existence ultimately depends on the charter schools or college sports programs looking good.
But — the New York City Independent Budget Office published a report in July 2015 that was pretty shocking. in terms of charter school attrition rates. It was ignored by the press, but I urge you to look at it. I am surprised more journalists have not jumped on this and requested more information from the IBO.
This report: “School Indicators for New York City Charter Schools 2013-2014 School Year” was purportedly a “descriptive” look at charter schools. But on Page 9, the IBO threw in what was the most important (and never before seen!) data in the entire report. Page 9 is the “Student Attrition and Backfill” report of 53 charter schools.
Instead of looking at one year’s attrition data, which can be manipulated by exactly the methods you mention — not including students who leave before October 7 or over the summer — this IBO report looked at how many of the starting cohort of Kindergarten students who won a random lottery left charter schools before they reached 5th grade. The results were pretty shocking — an “average” of 49.5% of the Kindergarten students who won the lottery and enrolled in the charter schools were gone! And of the ones remaining, an untold number had been held back — sometimes more than once — so that the 5th grade class could have significantly fewer than half of the starting Kindergarten class.
Here is the link to the report: (Page 9 has the attrition rates)
Click to access school-indicators-for-new-york-city-charter-schools-2013-2014-school-year-july-2015.pdf
There are 53 charter schools in this aggregate 49.5% attrition rate from K-5. Some are KIPP and some are Success Academy schools — the IBO chose not to rank those schools and thus the low attrition rate for KIPP charter schools may hide a much higher attrition rate for Success Academy. It is disturbing that the IBO did not list each of those charter school’s attrition rate separately, instead of lumping them together in a way that allows the charters who keep most of their students to hide high rates of charters who don’t. But I am certain the IBO would provide the disaggregated data to any journalist that asked.
The reason I am sure that the IBO could not refuse a journalist’s request for that disaggregated attrition data of those 53 schools is found on Pages 22 and 23 of that very report. There, the IBO went to great pains to disaggregated the PERFORMANCE (on standardized tests) data by charter school. In fact, in the report the students in charter schools were not really doing better IN AGGREGATE on the state tests. So the IBO chose to rank the charter schools individually to show that one charter school — Success Academy — had results that dwarfed every other charter school and put them to shame. Why wouldn’t they do the same thing when it came to ATTRITION rates? Why make a point of showing that despite the “average” performance of charter schools on state tests — which was very mediocre — one charter school far surpassed that, while leaving the attrition data aggregated so that it is impossible to tell whether the charter schools with the highest performance were also the ones with the highest attrition? Wouldn’t THAT be an important fact to find out?
Mr. Merrow, I hope you will request the disaggregated ATTRITION data from the 53 charter schools on the table in page 4 so we can see which charter schools exceed that 49.5% attrition for their starting Kindergarten class, and which charter schools keep most of their students. And whether the high attrition rate might correspond with high performance rates which the IBO very happily provided by individual charter school chain.
When the defenders of the “no excuses” charter schools say “let’s compare the attrition rates to public schools” we all know that is meaningless. Let’s compare the charter school attrition rates to one another! Charter schools — ALL of them — have parents who were motivated enough to sign up their kid for the lottery. Let’s see the IBO report’s ranking of charter school attrition rates just like they gave us the ranking of charter school performance rates. The IBO has the data. I hope you will ask them for it. Do the highest performing charters lose more kids that the ones who don’t get such good test scores? If so, something very wrong is going on.
LikeLike
I will try, and I will alert some journalists who are still in harness somewhere. Thank you for this detailed comment.
LikeLike
Thank you. I apologize for a typo I made — the attrition information is on page 9 of the July 2015 IBO report (not page 4).
The table on Page 9 of the July 2015 IBO report also contains backfill information. But as you clearly explained above, replacing a student you don’t want with a student you do want is not “randomly” educating all the students who win the lottery. The best measure of a charter school is how well they are doing with the starting Kindergarten cohort, not the children who may join the 1st grade AFTER the charter school tests them and decides that they don’t have to be held back a year, which dissuades many parents from accepting the lottery seat they won. NY1 ran a story back last fall about a young girl whose family complained because her previous public school wouldn’t accept her back after she left 1st grade because she had won the Success Academy Williamsburg lottery for a spot she planned to take. But when Success Academy tested her and told her she would have to repeat a year if she wanted to come to their school, the girl’s parents wanted her overcrowded public school to take her back. I wonder how many other students who get those “backfill” spots are tested at Success Academy and dissuaded from coming? So looking at the FIRST lottery winners — who could not be tested before joining Kindergarten — is the only way to measure whether a charter school is keeping or losing the randomly selected students.
How can anyone object to comparing charter school to charter school by attrition rate the way charter schools are compared by the test performance of their children? The only reason is if you have something to hide.
LikeLike
By the way, a ChalkbeatNY reporter — who since left — made some attempt to get disaggregated information about those 53 charter schools from the IBO last July when the report came out. You can see his article here:
http://ny.chalkbeat.org/2015/07/15/here-is-the-ibos-backfill-information-sorted-by-charter-network/#.VrOQ085fWoU
As you can see, the reporter did confirm that 4 out of the 53 charter schools in that IBO report with the 49.5% “aggregate” attrition rate were Success Academy schools. But when the reporter asked for disaggregated ATTRITION rates of the 53 schools, the IBO instead provided them with disaggregated BACKFILL rates! Why?
Why would the IBO hide the INDIVIDUAL attrition rate of each of the 53 charter schools so that is it impossible to know whether the highest performing charter chains were losing more of their starting Kindergarten students than they should be losing? Why would they instead only provide backfill information when we already know that a charter school can backfill with a more selective group of students? Backfill is nice, but it has nothing to do with the 49.5% attrition rate of the students in those 53 schools who won the lottery and started in Kindergarten. Were some charters doing a much better job of keeping the Kindergarten students who win the lottery and did some of those 53 charters lose a disproportional share of those 5 year olds when compared to other charter schools?
And this article raises the question of WHY a charter school needs to backfill so much of their class? Why are your students leaving far more frequently than the students in other charter schools?
I can’t imagine that anyone who purports to say that they care about metrics would not want to know the answer to that question.
LikeLike
[Again, it’s worse than ‘apples and oranges.’ Charter schools manipulate the data, legally, as I noted. They CAN and DO get rid of kids, but traditional public schools cannot do this. So the comparison would be meaningless. I am not trying to evade a question, just explain why the question is not the right one to ask.]
OK, but then what fundamental question are you trying to ask?
Say, you pick 2-3 problems with charter schools, and then I pick 2-3 problems with traditional public schools, and then what? What is the goal of that kind of exercise?
The fundamental, underlying argument from many SA critics often is (as I understand it) – “if you were to able to compare apples to apples, accounting for all the difference in the student demographics etc…, Success Academy would fare no better, than traditional pubic schools”. I obviously disagree with that conclusion, but I find that accurately executing the “apples to apples” comparison is a complicated technical task (something that I happen to be used to because of my profession), and therefore is easy to misinterpret.
If that is not the argument, then what is? Accounting rules for attrition between SA and the DOE are different? If you aren’t saying that, but then what are you saying?
LikeLike
I am saying (again) that attrition is not a meaningful statistic at traditional public schools, unless it’s really way off the charts, in which case it might indicate a problem with the adults there. Charter schools can dismiss students; traditional public schools cannot. Their attrition rate is generally a function of outside factors like family stability, parental employment, and so on
LikeLike
I would really like to call the bluff of these Success Academy parents. They aren’t pro-charter – they are pro-Success Academy. Anyone who is pro-charter would not want to see every other charter school in NYC look like it has such terrible administrators and teachers because they have “no excuse” for not getting the results that Success Academy has — after all, the administrators at KIPP and Icahn and Democracy Prep can fire their teachers at will. So why are their results not even close to Success Academy? Not just a little worse, but FAR worse? I think people like Yuri N. would probably argue that Success Academy should take them over and use their “secret sauce” to make sure those kids aren’t suffering at the hands of the mediocre charter school operators.
The IBO has the one decent attrition study that has been done that compares charter school to charter school. Fifty-three of them. They lost 49.5% of the 5 year olds who won their Kindergarten lottery IN AGGREGATE. Why wouldn’t everyone want to know how that breaks down by school?
LikeLike
[Success Academy parents …aren’t pro-charter – they are pro-Success Academy]
Parent010203 – that’s an astute observation. I am generally pro-Success Academy, that my child attends, but that doesn’t automatically make me a blanket pro-charter supporter. And, by the way, that doesn’t make me automatically love every simple thing that SA does – I like the school, but it has room for improvement.
I am not very familiar with other NYC charter schools, so don’t want to opine on things that I don’t know well.
I am still confused why you quote 49.5% student attrition over 5 years as if it were a problem, even though it amounts to a yearly attrition of about 10%, also undercutting part of John’s point, that summer attrition at SA is not counted at all.
I agree that I would love to see dis-aggregated attrition data by individual school, for SA, other charter schools, as well as traditional public schools. I am a data guy by trade, so I always want more data and data-drive analyses.
LikeLike
“I am still confused why you quote 49.5% student attrition over 5 years as if it were a problem, even though it amounts to a yearly attrition of about 10%, also undercutting part of John’s point, that summer attrition at SA is not counted at all.”
We have no idea whether Success Academy’s attrition is 10% a year. The AVERAGE yearly attrition of 53 charter schools was, apparently, about 10% a year. Just like the AVERAGE test scores of all those charter schools weren’t very good. It is only when you disaggregate that data that certain ones — like Success Academy — are huge outliers. We know that Success Academy schools are huge outliers for test scores AND huge outliers for suspension rates. (No other charter school suspends their 5 and 6 year olds like Success does.) The only missing data is whether they are huge outliers for attrition rates. As someone driven by data-analysis, you have to theorize that it is very likely that being a huge outlier for test performance and suspension rates goes along with being a huge outlier for attrition rates of the entering Kindergarten students.
What’s scary is that the AVERAGE charter school among those 53 is losing 49.5% of the Kindergarten class by 5th grade. So what does it mean to be an outlier? Losing 60%? 70%? And remember, the IBO includes all the students who didn’t even reach 5th grade but are still in the school, having been held back an untold number of times. Remember how long the child profiled in Mr. Merrow’s piece remained at Success Academy? Years. But he never got to a testing grade before they gave up on him. And blamed the child for acting out instead of looking at how their own methods might be one reason all those under performing students leave. Of course, if you WANT those underperforming students to leave, no need to examine your own teaching methods.
Of those 53 charter schools in the study, Success Academy has to be the most appealing both because it has millions of privately-raised dollars and good test scores. It should have the lowest attrition rate of all the charter schools in that study. If it doesn’t, then something is very, very wrong. Are you theorizing that it does? Perhaps, but the only way to know is to look at the IBO’s disaggregated data for those 53 schools, as we both agree. I hope a reporter is able to get that data soon.
LikeLike
[I am saying (again) that attrition is not a meaningful statistic at traditional public schools]
Great, so what is the bottom-line point of your article then? If, out of all the issues, you CHOSE to focus on the weeds of attrition calculations, in order to prove them unfair, then you NEED a comparison baseline, that you consider “fair”. At a minimum, you should be able to say, that SA’s accounting logic is unusual, and the industry standard is X.
Is your bottom-line argument that SA uses self-serving methodology to inflate its accomplishments? In that case, the main question would be – is that a minor inflation on the margins, or a major inflation (some data analysis would help here).
LikeLike
Why would a non-profit use a “self-serving methodology” to inflate accomplishments? Isn’t this supposed to be about education?
Any person who knows data analysis knows that the fastest way to raise an average is to drop the lowest score.
I don’t understand why you think that a charter school being able to suspend at will any 5 year old who they pretend is “violent” (that would be over 20% of them in some schools with mostly low-income students) can ever be equated with a neighborhood public school who loses students who move away? Neighborhood schools cannot control who leaves without jumping through many, many hoops. Charter schools – thanks to appallingly lax oversight — apparently can. Fortunately, few charters have been willing — so far — to use all the methods at their disposal when the parents of a 6 year old who doesn’t “fit” are not getting the message that they should find another school for the child who isn’t wanted.
Look, it would be a wondrous thing if Eva Moskowitz had figured out some magic way to turn all the at-risk students in failing public schools into scholars. That is certainly what she has been pretending to do for the last many years. Everyone knew the emperor had no clothes. Why else do you think that the pro-charter folks have suddenly changed their tune and are now claiming they are for the “strivers”? That’s the students who Success Academy has always wanted. The strivers, and when they didn’t find enough strivers among at-risk kids who won their lottery, the middle class students in Bensonhurst, mid-town Manhattan, the Upper West Side, and brownstone Brooklyn.
LikeLike