Highly respected NYC teacher placed in bottom 7% by Value Added method of scoring. A real shame.

[madfloridian]

This is the method that was used to publish the names of teachers in Los Angeles and call them failures or successes. They did not include evaluations and performance reviews even if they were excellent.

From an article by Michael Winerip in the NYT in March. This is the type of farce in evaluating teachers that is going on all over the country. NYC is leading the way.

Evaluating New York Teachers, Perhaps the Numbers Do Lie

Yes, it seems they do lie.


Chang W. Lee/The New York Times
Though her principal praised her work, Stacey Isaacson received a poor ranking in a statistical model used by New York City schools to evaluate teachers.

No one at the Lab Middle School for Collaborative Studies works harder than Stacey Isaacson, a seventh-grade English and social studies teacher. She is out the door of her Queens home by 6:15 a.m., takes the E train into Manhattan and is standing out front when the school doors are unlocked, at 7. Nights, she leaves her classroom at 5:30.

Her principal, Megan Adams, has given her terrific reviews during the two and a half years Ms. Isaacson has been a teacher. “I know that this year had its moments of challenge — you always handled it with grace and presence,” the principal wrote on May 4, 2009. “You are a wonderful teacher.”

On the first day of this school year, the principal wrote, “I look forward to being in your classroom and seeing all the great work you do with your students,” and signed it with a smiley face.

The Lab School has selective admissions, and Ms. Isaacson’s students have excelled. Her first year teaching, 65 of 66 scored proficient on the state language arts test, meaning they got 3’s or 4’s; only one scored below grade level with a 2. More than two dozen students from her first two years teaching have gone on to Stuyvesant High School or Bronx High School of Science, the city’s most competitive high schools.

But here it comes. Here is an example via the NYT of the infamous Value Added model of judging teachers. Just look at the graph.


A statistical model the school system uses in calculating the effectiveness of teachers.
By MICHAEL WINERIP Published: March 6, 2011

Impressive looking, but it does so much harm.

This may seem disconnected from reality, but it has real ramifications. Because of her 7th percentile, Ms. Isaacson was told in February that it was virtually certain that she would not be getting tenure this year. “My principal said that given the opportunity, she would advocate for me,” Ms. Isaacson said. “But she said don’t get your hopes up, with a 7th percentile, there wasn’t much she could do.”

The article points out that if the "mayor and governor have their way, and layoffs are no longer based on seniority but instead are based on the city’s formulas that scientifically identify good teachers, Ms. Isaacson is pretty sure she’d be cooked."

That's tragic.

There was another article from the NYT this week, about being defined by data. It's an excellent one.

In a Data-Heavy Society, Being Defined by the Numbers

Numbers and rankings are everywhere. And I’m not just talking about Twitter followers and Facebook friends. In the journalism world, there’s how many people “like” an article or blog. How many retweeted or e-mailed it? I’ll know, for example, if this column made the “most e-mailed” of the business section. Or of the entire paper. And however briefly, it will matter to me.

.."“For almost anybody in the United States under the age of 25, the only models are quantifiable rankings,” he said. So when students are researching a paper, how do they decide where to turn for the greatest expertise? Often, he said, by looking at what articles or papers online have the most hits.

The writer makes a couple more great points.

The obsession with numbers, he said, means we don’t trust or even look for the intangibles that can’t be measured, like wisdom, judgment and expertise.

We also lose a sense of ourselves as anything but a number and a rank, and start feeling bad if our numbers don’t measure up to others.

When a teacher like Stacy Isaacson who is considered outstanding by all who work with her is judged to be in the bottom 7% by a series of numbers...then something is very wrong.

And we are losing a sense of what education is all about, what makes a really great teacher. The saddest part is that it is all apparently being done purposely, deliberately.

If there is enough "failure" it makes it easier for "reformers" of all stripes to move in and take over...in the name of a pending crisis which really isn't there at all.

[ejpoeta]

1. I remember when I took driver's ed when I lived in Arizona. What we did was to

use simulators to 'drive'. I did great!! I did awesome!! When I took my final, I had to get a 38 to not get an A. I was the worst driver there was when I finally got my license. I got into a lot of accidents and it took me a long time before I became a not horrible driver.

My point is this.... You can test and use all kinds of bs parameters to judge someone's ability at something. But there is no test that will show you how good a teacher is beyond them actually doing it and gaining experience.

[jsamuel]

2. if i read that correctly, it looks like about 2/3rds of her score is based on her school and her

district, not on her or her classroom's performance.

[Recursion]

47. No

That's not what it says at all. Her 'score' is one of the alphas, and it's found by solving the system of linear equations that sets up.

[jsamuel]

94. I see now

Are the classroom, school, and district variables treated as if they are independent of each other?

Also, how does this account for a teacher working with a particularly difficult group of students? They do have the "student characteristics" score. It only compares the pre-test and the post-test scores. I can tell you there are some classes by where I live that obviously have some very troubled students (can tell because they are that way through multiple years) while other classes have higher level students. While some students are likely to move, others are going to be difficult to move. Some of the better indicators I have seen only treat the student scores as 1/3 of the overall teacher rating. They use observations for another 1/3 and the overall school score for the other 1/3.

[Recursion]

97. I'd have to see the model to answer specifically

The weights of the classroom, school, and district performance are the I coefficients; they are found by some more or less (probably less) meaningful heuristic regression over past data (this is difficult to do well for widgets, let alone students).

I'm not meaning to come across as saying this formula is a good idea; my point is just that it's well known from systems theory and has had useful applications before, even in "soft" situations (the wargames that convinced Shinseki we needed 400k troops for Iraq, for instance). In general this works better on large scales than small ones.

[Brickbat]

3. So many thoughts when I read this.

1. While I'm all for statistical-based analysis, that graphic makes me sick when I look at it.

2. "There wasn't much the principal can do." Well, maybe that's true. There isn't much that the principal can do that's easy or protects her job and still does the right thing by the teacher. If there's anything Wisconsin should be teaching us, it's that the other side is making the right thing difficult, unpopular and sometimes illegal. That doesn't mean we should stop doing the right thing.

3. The point about obsessing with numbers and losing sight of intangibles is brilliant.

Thanks again, mad, for all your hard work here and IRL.

[madfloridian]

11. You are right about the principal. She should have stood up for the teacher.

It just shows the principal fears for her job also.

It's really gotten to be a mess.

I am very angry our Democratic leaders refuse to speak out when attacks are made on public education. They all appear to have fallen for the propaganda about how bad teachers are...and that public education is failing.

I am very angry.

[Citizen Worker]

158. Many, if not most, democrats support this garbage and are for charter schools, vouchers and

ultimately total privatization.

[eppur_se_muova]

4. Is there *anything* to support that model besides the *assumption* that it works?

There seems to be a built-in assumption that all these factors can, and have been, successfully deconvoluted from one another, and that no factors have been overlooked. Furthermore, they are reporting data derived from test scores to three significant figures. No one in their right mind considers any written test to have that sort of confidence level. The SAT has a reliability of ~0.9, and it's been studied and refined more than just about any test out there.

[madfloridian]

5. What they leave out is humanity, individuality, quality and depth of learning....

and I could go on for hours. I look at that method of testing, and I see a lack of knowledge of what children are all about...of their personalities and their character. A test score has become too definitive.

The testing companies are reaping great benefits from this mindset. The teachers, kids, and parents are getting tummyaches and bad cases of nerves.

[callous taoboy]

110. Exactly! What other nuanced profession (e.g. counseling)

could weather such a square assessment of success? None. Teaching, like counseling, is a bit magical if you will. There are so many nuances at play in the classroom, many of which could never be picked out on a standardized test.

[Turbineguy]

6. When the goal

is to destroy education, the good teachers are the ones to get rid of.

[Jakes Progress]

7. How many times can we read about what Obama

wants our schools to look like, what grover norquist hopes our schools will be like, what new gingrich works to make our schools like before we get it?

Are we going to get it? Or is education a cause lost to the NeoDem takeover of the Democratic party?

[thanks_imjustlurking]

8. That formula appears calculated to confuse, discourage questions, and intimidate.

Reminds me of the time not too long ago when nobody (almost) questioned collateralized debt obligations, credit default swaps, etc., because they were supposedly too complicated for ordinary mortals to understand. Trust us, small people.....

[madfloridian]

9. It does, doesn't it? It makes no sense to me at all.

And I agree it is geared to make us feel unworthy.

[Recursion]

52. In all seriousness, what's confusing about it?

The final test is modeled as a sum of:

1. The previous test
2. A model of the student's aptitude
3. A linear system involving the classroom, school, and district (you solve for the "classroom" element here to get the teacher's "score")
4. An error factor

It looks like every single systems theory equation I've ever seen.

[thanks_imjustlurking]

98. I guess, although I think it would look intimidating to many.

But I'm sure my view is skewed by the fact that I don't think helping kids is really the goal of these people and their equations.

Edited to add that I also think it's interesting that the teacher's students were so successful yet that formula put her in the bottom 7%. I guess the moral of that story is "don't teach students who would probably do fairly well with anyone but a total incompetent, and who have little room for improvement because they're already at or near the top." (At least that what it looks like to me happened to her.)

Oh, wait - public school teachers don't get to choose their students, any more than public schools do. It's a heck of a note when you get fired for getting the good students.

[Recursion]

112. She's not getting fired

She isn't getting tenure and probably won't without teaching more "needy" students. Yes, it's an incentive to get teachers to go to more "troubled" schools -- that's a feature, not a bug.

[thanks_imjustlurking]

170. As I understand it, she doesn't have a job anymore due to the formula.

That sounds like "fired" to me.

I agree about encouraging good teachers to work at tough schools, but I don't think terrorizing them is going to do it. I think it's just going to drive good people out of teaching. You *do* know the five-year attrition rate even now, right?

[Recursion]

172. Maybe I read it wrong; she's predicting the next round of lay-offs will include her

Which sucks, but that's why we should tax people at a level where we don't have to lay off teachers.

I agree about encouraging good teachers to work at tough schools, but I don't think terrorizing them is going to do it. I think it's just going to drive good people out of teaching. You *do* know the five-year attrition rate even now, right?

It's incredibly high nationwide, and I believe even higher in New York. Maybe if people didn't demonize the TFA kids so much they would stay longer.

[thanks_imjustlurking]

173. The TFA kids?

I'm not doing any more research on it right this minute, and I'm up for being proved wrong, but my current impression, based on what information I've assimilated so far, is that most of them are doing it for their resumes and have no intention of staying longer than the minimum.

[Recursion]

175. I was being snide

I don't have any data on the motivations of TFA participants. I guess I do wish a lot more people taught, for shorter periods of time, than is the model now (the only work I've done with that is on the other end, getting school systems to let retiring scientists and engineers teach -- the resistance to that really bugged me and I should probably just let that go).

[joshcryer]

199. Their primary goal now is retention rate, I have a friend who is with TFA.

I think the TFA demonization is fucking bullshit. Just saying they "don't want to teach but want resumes" is really insulting, imo.

[thanks_imjustlurking]

201. Very interesting, then, that one of the main tabs on their own website is

"After the Corps."
http://www.teachforamerica.org/

And I quote:
Our Alumni
Alumni Careers
Alumni Support
Grad School Partnerships
Employer Partnerships
Alumni Summits
Alumni Leadership

Further: 46,000 applications for just 4,400 teaching slots? Call me cynical, but I don't think that's *just* people *just* wanting to "make a difference for the kids." Some, sure, but 10X the number of applicants for slots?
http://jaxkidsmatter.blogspot.com/2011/01/23-of-13-keep-teaching-after-two-years.html

Found a mediocre critique, but the comments are quite interesting. From a TFA alumna:

Me personally? I probably won’t stay in teaching. Why? Because I don’t have enough supplies for my students. Because standardized testing is ruining education. Because I’m expected to come early, stay late, and work on weekends, for crappy pay. Because I have 30 students when I’m supposed to have 22. Because its 10 times more stressful than any office job I’ve ever had. These are problems we need to fix on a more fundamental level, rather than just blaming teachings for leaving a terrible situation.

Lots more in the comments - scroll down.
http://www.feministe.us/blog/archives/2008/08/23/why-i-hate-teach-for-america

This says what I was thinking after having a look at TFA's website. Again, it's in the comments.

After looking at their website, the program sounds less focused on the underprivileged kids and more focused on cultivating future generations of leaders and politicians who will become education policy decision makers. That reeks of attempting to generate support for the program's funding in an unending, incenstuous cycle. They tout that in 2010 alone, they cultivated 554 school leaders, 45 elected officials, and 8 social entrepreneus from their alumni base. So, in addition to asking where the money goes, I am also interested in learning why the organization is so obsessed with using these underprivileged kids as a step-ladder for furthering the career objectives of their members.

http://www.nola.com/education/index.ssf/2011/01/teach_for_america_gets_100_mil.html

[thanks_imjustlurking]

202. Ha! Again the best is in the comments.

Oh, BTW, joshcryer, if you are a teacher I hope you do not use such language in your classroom....

Why not an Executives for America-EFA
Why not fire all those multi million executives and replace them with newly minted MBA and marketing grads.
Even if paid $100,000 per annum as a starting salary, they would still be a bargain. If anyone can teach, anyone who has an advance degrees in finance, business, accounting etc. should be able work as a corporate executive. After two or three years they could be replaced by the new crop of finance grads.
What a wonderful way to save money and increase profits

http://www.usatoday.com/news/education/2009-07-29-teach-for-america_N.htm

[joshcryer]

198. Oh, DU is a strong believer in "tenure after years of service" regardless.

You could be the worst teacher ever, you deserve tenure. The kids be damned.

[theaocp]

10. If you're into college football,

you're probably aware of the horrifying inequities with letting a computer into the rankings. Aye, BCS, I curse thy name. This is similar and absotively terrifying. I wonder if our best and brightest educators threatened to "take their talents elsewhere" like our cowardly CEOs and banksters do, would they suddenly have politicians fawning over them instead of denigrating their efforts? C'mon! Laugh along with me! :rofl:

[bertman]

12. The fact that any Democratic administration could push this makes me SICK. REC. nt

[madfloridian]

13. I have been very angry about it.

I think of all the years I taught and was so proud of it.

They are taking away from present teachers the ability to feel pride in their work, and it makes me very resentful.

[Overseas]

14. Very sad and so dehumanizing.

Thank you for sharing this story with us.

[saras]

15. Textbook example of the failure of the 'value added' model

If you actually THINK about what the model says, it's pretty obvious that it's a piece of shit.

And since it demonstrably doesn't measure what we want in teachers, AND allows bad teachers in bad situations to measure highly, it seems to serve no useful purpose at all.

Except for, of course, the dismantling of public education without honest debate and discussion...

[proud2BlibKansan]

16. I am not aware of any research that supports the value of this value added crap

Why doesn't that matter?

[Recursion]

55. It's pretty basic systems theory

I'm not sure what research you'd be looking for; all processes with feedback work that way. If you mean the lambda and zeta terms aren't measured with sufficient accuracy (and are probably in fact non-linear), then that's probably certainly true, like it is in any field, which is why we only linearize over a relatively short period.

[SlimJimmy]

56. Actually, "Value Added" is a Lean Six Sigma construct and is

widely used in industry to streamline production and services. When looking at a process map, "Non-Value Added" is stripped out in an effort to make production more efficient.

[madfloridian]

67. "widely used in industry to streamline production and services"

Really great for education...streamline those kiddies. :sarcasm:

My mind is boggled at the lack of understanding that children are not products.

:shrug:

[SlimJimmy]

72. Education is a "service", so the value added formula can be apllied to it. Think of

it as an efficiency model for service oriented products. Now, that's not to say that this particular model may be flawed in how it determines what is value added. For example, I'd like to see how they internally calculate some of the factors they used.

[madfloridian]

73. "as an efficiency model for service oriented products."

Baloney

[SlimJimmy]

84. Not "baloney", statistical analysis used in industry every day. I guarantee the vehicle

you drive was designed,improved, and produced using these very same methods.

[madfloridian]

93. I taught real live children. This is about them, not vehicles.

[demodonkey]

107. [phone] nt

[SlimJimmy]

139. If you don't or won't try to understand LSS, then there's nothng more I can say.

[thanks_imjustlurking]

171. I understand it fine.

Don't do things that don't add value. Do do things that do. Edited to add that I try to do that, on an individual and sometimes improvisational level, every day of my life.

What I don't agree with is that the people who are, IMHO, out to destroy the public school system and profit off education have found a way to measure that with regard to kids. At least not with their GIGO formula.

[SlimJimmy]

182. I agree. I have a great deal of faith in the LSS system, but I have some

major issues with how this particular formula was derived; and I have said so several times in this thread. If *some* don't care to see that, then it's their loss.

[madfloridian]

178. Fine with me.

[SlimJimmy]

183. I'm sorry to have moved your cheese. I know how upsetting that can be. (nt)

[proud2BlibKansan]

120. Children are not cars

Why is this so fucking hard for the bashers to grasp??

[SlimJimmy]

141. Teaching is a service and can be made more efficient using LSS. What

is so hard to grasp about that? For example, taking attendance at the beginning of class is a non-value add to teaching. I would automate the process at the door and give that time back to the teacher to do something value added ... like teaching. I can think of a lot of non-value add in a teacher's day.

[Recursion]

143. No, but they vary measurably across the population of students

And their improvement over the course of a given year is modeled as a combination of development (zeta), previous attainment (lambda Y sub zero), the student's effort (beta X sub i), noise (epsilon -- was it too hot in the room that day? Did the kid's parents have a fight the night before? etc. -- maybe the epsilon should be much higher than it is), and a 3x3 linear system of the teacher, school, and school district.

Maybe students can't be objectively assessed on subject mastery. In that case tests should govern a lot less of the course of their lives than they end up doing.

If they can, some of their performance has to come from the teacher they had -- if not, why do we care who teaches?

This model sets up the 3x3 linear system to determine how much a students' performance comes from the district (by looking at how all the students in the district did), how much from the school (ditto), and how much from the teacher. And, again, if none of it actually comes from the teacher, why do we care who's teaching?

[riderinthestorm]

17. It's too late in the eve for me to comment but let me just K&R this back to the top.

I can't believe that equation functions in real life. It's odd, my 14 year old brings home a lot of fantasy/sci fi to read. And I always read it along with her, picking up the books when she's working on her homework or in the shower, or riding or whatever. And right now she has a mythology fiction spin-off and I swear, I had a moment of transportation looking at that equation coming off reading a chapter in her book.

I have to get myself back to my murder/mystery novel in order to ground myself in reality again. :rofl:

Your excellent OP makes me want to laugh and cry....

[chervilant]

18. Succinctly:

When a teacher like Stacy Isaacson who is considered outstanding by all who work with her is judged to be in the bottom 7% by a series of numbers...then something is very wrong.

Our species seems hellbent on wallowing around in the muck and the mire at the bottom of our self-immolation abyss...

[truth2power]

19. Madflo, I tried everything to print that formula to show it

to someone who knows statistics. Can't do it.

Does the NYT do this on purpose?i.e. making it impossible to print?

[madfloridian]

21. Right click it, save it, and print it from your computer.

I would help but it looks like Greek to me, a veteran, retired elementary teacher who is smart but not good at formulaic stuff.

[Recursion]

60. Really?

A result as a weighted sum of inputs is that foreign?

[trud]

20. you certainly don't want to actually measure things

I mean, how crazy is it to do that.

I'm glad to see that there is an effort in place to determine how effective a teacher is before they're handed a key to a lifetime job. How many of us in the private sector get that kind of guarantee after three years?

There's a lot of emphasis in the article about her showing up early and so forth. That doesn't matter a jot if the kids aren't improving relative to kids in other classrooms.

[chervilant]

25. Do you REALLY want to talk about 'crazy'?

The Lab School has selective admissions, and Ms. Isaacson’s students have excelled. Her first year teaching, 65 of 66 scored proficient on the state language arts test, meaning they got 3’s or 4’s; only one scored below grade level with a 2. More than two dozen students from her first two years teaching have gone on to Stuyvesant High School or Bronx High School of Science, the city’s most competitive high schools.

Surely there's a course or workshop near you that could help improve your comprehension skills? Individuals with a propagandized perspective of teachers and 'tenure' are part of this problem, trud. Your time and energy would be better served advocating for genuine reform in public education, rather than partisan attacks on teachers and unions.

[proud2BlibKansan]

26. +1

[JackRiddler]

64. What a nuts system.

Basically she's punished for her or the school doing so well that her students couldn't improve very much off the level they've reached.

THIS IS INTENTIONAL.

[Recursion]

65. She isn't being punished, she just isn't being rewarded

She's not getting tenure. That's the downside of teaching the "easy" students.

[JackRiddler]

68. Despicable propaganda from you.

Being kept in economic insecurity and threatened with layoff is a punishment. If that's normal, by the way, then it doesn't become right, or less punishing. If that's normal, it's an indictment of the system as a whole.

Getting to teach the "easy" students is not and cannot be seen as a compensation for this treatment. Only someone who is remote from the welfare of children (whether or not you deal with them personally) could say such a thing. There are no "easy" students and teaching them should not mean precarity or being treated as though you're a freeloader.

Furthermore, they appear to be "easy" in the first place because the school and she (by ALL appearances) are good. So yes, she is being punished for being good at a good school, NOT on behalf of any worthy goal, but in the context where the Billionaire Boys and the education deformers are out to bust the labor. And you are coming up with the lame-ass black-is-white rhetoric that is supposed to excuse this attack on teachers in the name of "improvement."

[Recursion]

69. Well, I'm sure she can transfer to a more difficult school if she wants tenure

It's not like she's getting fired.

And I suffered through way too many horrible teachers who had ossified themselves into a permanent fixture in their classroom to lament the possibility of higher turnover.

[JackRiddler]

70. Tenure is not a carrot for the corporate mentality to dangle in front of labor commandos.

This is teaching. Tenure protects the profession and assures that the good people can be retained in ALL areas. Teaching "good" students is not a special privilege. Teaching "difficult" ones is not a punishment for which you get tenure as a reward.

Your reductive, corporate-borrowed quasi-Darwinist vocabulary speaks to nothing that relates to schools, teachers, children or learning.

[callous taoboy]

167. + 1,000,000

[chervilant]

140. Perhaps,

that explains your lack of compassion for the thousands of teachers nationwide who are being RIFfed or fired, allegedly because of 'poor performance.' What--did hundreds of administrators (finally?!) wake up and smell the coffee?! OR, are these teachers the victims of the corporatists' assault on public education?! Frankly, this is not a case of 'you decide'!

Until and unless you become an informed advocate for an improved system of public education (at the very least warranted by your 'way too many horrible teachers' experience), I would strongly encourage you not to post such indefensible drivel.

[JackRiddler]

195. You tell'em!

[proud2BlibKansan]

122. Why would you assume her kids are "easy"?

Once again, you have no clue.

[Recursion]

125. Because high-performing kids are in a lot of ways "easier"

Yes, it's harder in some ways, but a magnet that requires a test for admission isn't going to have nearly as many kids who are showing up not having eaten anything in the last 24 hours but the tater tots from lunch yesterday. Or whose parents are pressuring them to drop out of school and work at the gas station they own. Etc.

[Recursion]

66. First people complain that plain testing punishes teachers of poor students

Now a more-sane system that involves student progress fails to reward teachers of already-excelling students (which makes a lot of sense).

If we aren't to say that teachers are completely purposeless, we have to say some of them are better than others, and that how students perform after their time with the teacher is in some way a proxy for that difference.

[chervilant]

132. hmm...

While it's true that teachers run the gamut from very, very bad to very, very good--just as do members of any other profession--the vast majority of teachers are fully capable of performing the teaching tasks for which they're responsible.

The current administration inexplicably follows in the footsteps of the Bush administration--and the grand tradition of the corporatists--with regards to vilifying teachers and unions. This assault on teachers is an indefensible corollary to privatization for the express purpose of making 'public' education profitable to certain corporatists.

[Recursion]

135. Of course they are

While it's true that teachers run the gamut from very, very bad to very, very good--just as do members of any other profession--the vast majority of teachers are fully capable of performing the teaching tasks for which they're responsible.

Certainly, nationwide. I've seen horrible districts, though.

This assault on teachers is an indefensible corollary to privatization for the express purpose of making 'public' education profitable to certain corporatists.

I'm still not sure what a "corporatist" is, so I can't comment on that, but I agree that Duncan has been disappointing in his (non-)defense of unions. But, then again, I've seen districts outsource a lot of their curricula to education management companies, and the simple fact that someone is making a profit at some point in the process does not automatically make it bad. (Nor does it by itself make it good.) And, for that matter, the most popular model of privatized education, private schools, are almost universally run not-for-profit.

[chervilant]

138. FYI

A corporatist is any individual who participates in the control of a state or organization (eg, our government, the media) by a large interest group (eg, the Corporate Megalomaniacs; aka, the PTB, the MIC, the uber wealthy).

Furthermore, you are missing the point. While supporters of privatization promote the meme that they are genuinely concerned about improving education for our children, making public education profitable is their true raison d'etre. AND, they are pursuing this end to the measurable detriment of veteran teachers across the United States. I encourage you to become informed about corporatists and about their ongoing assault on teachers and unions.

[Matt Shapiro]

176. I question two aspects of your support for this method of evaluating teachers

1. This value added method would seem to be more applicable to absolute measures like the number of automobiles produced per unit of time than to relative measures. I believe the test scores that are at the heart of this analysis are not only relative, but grossly relative -- quartiles. A group of students were in the top quartile at the beginning of the year, and remain at the top quartile at the beginning of the next year. Does that mean they made no progress? They may have learned an immense amount of information and their reasoning skills may have improved dramatically, but they will still be in the top quartile. The method is flawed.

2. Why do we assume that the standardized test scores have any validity? Because the testing companies say so? It serves them to say that. They cannot be trusted. My experience is that the tests measure only a small portion of the knowledge, skills and abilities that one can use in life or that are valuable to society. If the raw scores have little validity, the relative scores are next to meaningless. Basing teacher evaluations on changes in nearly meaningless relative student test scores makes no sense, other than to enrich the testing companies.

I hope you get to see this and respond.

[Recursion]

190. Two good points

. A group of students were in the top quartile at the beginning of the year, and remain at the top quartile at the beginning of the next year. Does that mean they made no progress? They may have learned an immense amount of information and their reasoning skills may have improved dramatically, but they will still be in the top quartile. The method is flawed.

Flawed (like any method) but not pointless. And I'm not convinced it's worse than observation as compared to a prior theoretical model of what teachers should do.

Why do we assume that the standardized test scores have any validity? Because the testing companies say so?

My worries there are much less about the teachers whose jobs that can impact and much more about the kids who have the course of their lives set by those test scores.

[proud2BlibKansan]

121. +1

[PVnRT]

27. So, let's say her class is performing well, which it is

And performs at that level consistently, than the metric will only show that there is no "improvement." How would you like to face being fired if you don't show "improvement" on your performance evaluation because the scores remain consistently good?

"lifetime job" - Really? When were teachers suddenly guaranteed lifetime jobs? This is news to me and everyone else who knows that the standard right-wing propaganda regarding teachers is complete bullshit.

[Yo_Mama]

31. Right

In theory, the worst possible student for such a teacher to have would be a student who scored perfectly the year before. There is no possibility of raising the performance!

Most of us would be thrilled to have a second year of perfect scoring, and would think that both student and teacher had done a great job. But in this system, the teacher would be a failure. I cannot imagine a worse way to inspire teachers.

A sad story but a great example of why modelling has such distinct real-world failures.

[progressoid]

48. +1

[Recursion]

41. Welcome to the private sector

How would you like to face being fired if you don't show "improvement" on your performance evaluation because the scores remain consistently good?

Yup. Happened to me a couple of times (though on more objective measures than performance evaluations). That's when it's time to move on.

[PVnRT]

57. Well, I guess it's OK, then

I mean, who cares if it's right or wrong. It's happened to you, so it should happen to everyone.

[Recursion]

58. Avoiding that is not a precondition to doing a job well

Was my point.

In the case of this teacher, she took the "easy" (ie, already high-performing) kids. Why shouldn't it be harder for her to get tenure?

[thanks_imjustlurking]

100. "she took "

Are you under the impression that public school teachers get to choose their students???

[Recursion]

115. You don't wind up in a closed-admissions magnet by accident (nt)

[proud2BlibKansan]

123. That's no proof that teachers are allowed to choose their students.

[Recursion]

124. No, but you have a lot of say in what school you end up at

And just looking at the formula one of its purposes seems to be to give an incentive for teachers to go to "hard" schools.

[proud2BlibKansan]

133. So? Lots of teachers can choose where they work.

That's completely different from choosing their students. I've taught in traditional public, magnets, a charter, private and Catholic schools. Never once have I been allowed to choose my students. It just doesn't happen.

[Recursion]

134. You did by which school you were at

Like SlimJimmy pointed out above, this is a weakness of this model in that it only rewards improvement and not maintenance. So, that's a factor teachers will have to consider when they're deciding what schools to go to, if they keep this model. And I don't see how that's worse than evaluation-based systems.

[callous taoboy]

168. Yeah, really. This guy really knows his stuff, right?

Your tenacity in arguing with fools is always appreciated, proud.

[proud2BlibKansan]

118. +1

[JackRiddler]

71. "Private sector" is not an acceptable euphemism for the rule of stupid cannibal logic.

If that's the "private sector," then the "private sector" is what needs to change.

When are you joining the Tea Party counter-protests in Wisconsin? Put your feet where your ideals already are.

[Recursion]

74. *yawn*

When I worked with DC Public Schools (this was pre-Rhee), 95% of teachers were rated "Highly effective".

Whatever flaws this system has, it's better than that.

And, FYI, I believe in collective bargaining, and pretending I'm a teabagger is A) against DU rules and B) shows the weakness of your argument.

[JackRiddler]

75. I'm not pretending, I'm pointing to where you're heading anyway.

So what did you do when you worked with DC public schools?

[Recursion]

78. I was a resource instructor for math and comp sci

Under a program (now-chancellor) Henderson started in the early oughts.

[SlimJimmy]

77. With a name like Recursion, I'm going to guess that you

might know a thing or two about statistics. :)

[JackRiddler]

85. Statistics are only as good as the model and the data used.

In this case, neither are relevant, both are chosen with a goal other than student welfare in mind.

[SlimJimmy]

88. That's true. But in this case, we don't have enough information yet to make an informed decision.

[Recursion]

87. This isn't even a statistical process

It's just a simple result-as-output-of-linear-system. It's kind of vague to the point of pointlessness, I just don't see that it's worse than doing things based on any other particular system (say, how your superiors and coworkers like you, or how well your students perform on a test).

[SlimJimmy]

89. I'll disagree slightly and say that I do see a statistical process in play. Agreed,

this is a linear equation, but it has statistical factors involved. Not really all that different from using a scatter plot to look at trends.

[Recursion]

90. Presumably the lambda, epsilon, and zeta come from a stastical analysis

So, ok, yeah, stats were used in making the model, but that part is completely opaque to us from the image.

[SlimJimmy]

92. Right, and I have said elsewhere that I'd like to see some of those internals.

[FLPanhandle]

30. Teacher effectiveness should be measured

However, the formula used should be subject to debate.

Even in the private sector, metrics are constantly evaluated for sanity and changing environments. If she truly is a good teacher, then somewhere the formula isn't capturing it or irrelevant metrics are skewing the results.

[SlimJimmy]

80. Agreed. And as I have said elsewhere in this thread, I'd like to see some of the

internal calculations that were used for some of the formula's metrics.

[CreekDog]

32. "lifetime Job" what the hell does that mean?

to teachers who have been laid off?

:wtf:

(but i don't even know why i'm arguing. if i'm arguing with someone who is against teachers (to the extent they can lie about them), am i even arguing with a Democrat?)

[TheKentuckian]

33. What is it that you think is being measured and in what context?

You see no benefit in having less room to improve? Why is moving from a 64 to a 72 more meaningful than maintaining a 97?

Her students are already tops, they don't have the space to improve like other students. It is a stupid measurement.

Give me a perennial MVP over most improved any day.

[SlimJimmy]

82. And therein lies the rub. There does not appear to be any value add for

maintaining excellence, only for improvement. So, an excellent teacher, with above average students will suffer using these metrics.

[TheKentuckian]

99. Consequentially, it is a stupid measurement. (nt)

[LanternWaste]

45. I imagine an objective measure based part and parcel...

"you certainly don't want to actually measure things..."

I imagine an objective measure based part and parcel on actual performance, in addition to other, relevant criteria would be more than acceptable.

(although I do realize that would deny many critics their daily melodrama)

[Angry Dragon]

83. I hope you find yourself soon

:thumbsdown:

[sulphurdunn]

96. Teaching is about measuring things.

Lots of things. We measure academic progress, social development, maturity and character. We only use analysis to measure academic progress relative to lots of intangibles: things like how little sleep some kids get, or whether they eat anytime other than at school, or whether mom's boyfriend has been using them as a punching bag or worse, or if the utilities have been shut off, or why they're sad, or happy, or hopeful, or have any faith in the future of the world we adults have created and are making worse. At least a quarter of all school age children in the land of the free live in poverty. How do you quantify that other than as a gross statistic? What does it mean?

Teachers don't actually get "tenure" like a college professor might. They get continuing contracts, which means it is not necessary to renew them every year. We don't have "guaranteed jobs for life". I'm really getting weary of hearing that meme.

I'm also sick and tired of hearing the "in the private sector we don't get..." analogy. The private sector has totally fucked up this country beyond all recognition and would be better run like the public sector. Worship of the private sector is no more rational than any other form of idolatry.

As for quantification being the be all and end all of every measure of success, know this: People are not widgets. Kids are not standardized products and neither are their teacher, and neither are you. Maybe the widget system works for producing things, but it doesn't work for educating citizens. It isn't even the black box assessment that is so offensive as it is the incomprehensible, punitive nature of it. The folks who design these measure have been unanimous in asserting that they should not be used alone to determine teacher competence or student development. Politicians have decided to use them for that.

You need to get over the idea that our country is some kind of assembly line, and that people are just another kind of quantifiable stuff that have numerical values. Free people don't think like that. There is far and away too much variation in the human population for analysis by formula without human subjectivity being the determining variable.

[Recursion]

117. Teachers are more than happy to set the course of kids' lives by the assessments they give

Why is it so intolerable to have a similar standard applied to the teachers themselves?

[sulphurdunn]

129. It's simple

If you want to use a quantifiable method to assess someone you don't use the assessment you give someone else to do it. Student scores on standardized tests are Mad Hatter measures for assessing teachers. If you want to assess teacher you assess teachers, not students. You would create a test that measures the teacher's content area knowledge, pedagogy and classroom management skills. The fact that this is not self-evident supports my contention that endless, mind numbing exposure to mass media over several generations has created a de facto Idiocracy despite the best efforts of educators, who ironically are being blamed for it.

[Recursion]

131. March hare

Student scores on standardized tests are Mad Hatter measures for assessing teachers.

They're good enough to determine much of the course of the students' lives.

If you want to assess teacher you assess teachers, not students. You would create a test that measures the teacher's content area knowledge, pedagogy and classroom management skills.

While ignoring whether that content area knowledge, pedagogy, and classroom management skills are actually making kids competent in the subject being taught?

Can students' competence in a subject taught to them be assessed with some degree of objectivity?

If so, why can that not be used as a way of assessing the teacher?

And if the answer is because the students exhibit way too much variance for that to be meaningful (and, you'll notice, the VAM here is an attempt to address that), why does it matter who is teaching? If it's really beyond the power of a teacher to reliably make their students competent in the subject, why do we care who's teaching at all?

[sulphurdunn]

152. Let me try a real word situation for you.

A special education teacher has a resource class for students with intellectual disabilities. They are taking Junior English. A general education teacher down the hall is teaching an honors class in the same subject and a third teacher is teaching a regular class in the subject also. All students taking junior English are obliged to take the same standardized test. Scoring is the same for all three groups. All of them must pass this test to receive a high school diploma. Failures to pass are counted against the school and the teacher.

All of the students in the resource class plateaued in reading ability before the 4th grade. They will never be better readers than they were then. That is demonstrably true. They will all fail the test. That is also demonstrably true. The honors class will all pass the test with ease, but their overall reading improvement will be slight. About 85% of the regular education students will pass. There will be a wide range of reading improvement or decline in this class.

Using the VAM, both the special and honors teachers will be found wanting. The regular teacher may or may not depending on how the scores shake out. In all three cases there is a far greater chance that any particular teacher will be found inadequate than otherwise.

In 2014 all three categories of teachers will be required to have 100% pass rates under NCLB regardless of the VAM analysis, or they and their schools will be considered failures. If a single student fails in any subject area assessed by a standardized test, even if everyone, including the student who failed is accepted to an Ivy League college, the school and all the teachers will be considered failures and will be subject to a broad range of bad consequences. Public education reform is a Mad Hatter system. That's the point I've tried to make. New types of "assessments" will only exacerbate what is already accelerating down the rabbit hole.

[Recursion]

154. You definitely can't get me to defend No Child Left Standing

It's a horrible plan.

A special education teacher has a resource class for students with intellectual disabilities. They are taking Junior English. A general education teacher down the hall is teaching an honors class in the same subject and a third teacher is teaching a regular class in the subject also. All students taking junior English are obliged to take the same standardized test. Scoring is the same for all three groups. All of them must pass this test to receive a high school diploma. Failures to pass are counted against the school and the teacher.

All of the students in the resource class plateaued in reading ability before the 4th grade. They will never be better readers than they were then. That is demonstrably true. They will all fail the test. That is also demonstrably true. The honors class will all pass the test with ease, but their overall reading improvement will be slight. About 85% of the regular education students will pass. There will be a wide range of reading improvement or decline in this class.


Yes, it's one of the many ways SPEd is broken, despite the heroic efforts of the people doing it.

So, why are the honors and SPEd student taking the same tests? For that matter, if we really know the SPEd students plateaued before the 4th grade, why are they taking high school English?

In 2014 all three categories of teachers will be required to have 100% pass rates under NCLB regardless of the VAM analysis, or they and their schools will be considered failures.

And that's not even the dumbest part of the law!

[sulphurdunn]

169. Becasue we're all standrardized

"So, why are the honors and SPEd student taking the same tests? For that matter, if we really know the SPEd students plateaued before the 4th grade, why are they taking high school English?"

It's an Orwellian sort of thing thing done in the name of equality. According to the reformers, every child deserves an equal opportunity to succeed. So, in essence, we compel these kids to attend school for 13 years and then throw them out the door with nothing to show for it because they failed at their impossible chance to be equal when they failed their end of course, all or nothing, high stakes standardized tests.

Now, to be fair, we do have lots of modifications for these kids when they take these test, but they fail anyway because kids who can barely comprehend a 4th grade reader are unlikely to analyze grade 11 level literary text even with accommodations.

[chervilant]

145. hmm...

Better to remain silent and be thought a fool, than to speak and remove all doubt...

When you make a statement like 'teachers are more than happy to set the course of kids' lives by the assessments they give,' you are tarring an entire DIVERSE group with a single simplistic and misleading assertion. This is similar in tone AND intent to "they all like to eat watermelon and fried chicken"! And, it's just as offensive.

When I teach, I am solely invested in finding whatever methods work for ALL of my students to give them the opportunity to have "AHA!" moments as they learn to play math, the oldest game we humans have created. Those "AHA!" moments are the big brass ring for me--and for countless other teachers across this nation.

If you think otherwise, perhaps you have need of a few "AHA!" moments yourself.

[Recursion]

146. OK, where's the teachers' movement to do away with testing?

Not to do away with student testing as a way to evaluate teachers, but to do away with testing as a way to evaluate students?

Those "AHA!" moments are the big brass ring for me--and for countless other teachers across this nation.

Yes, I love them, and they're why I tutor. But I leave the testing to the "real" educators, which "real" educators constantly remind me that I'm not, whether I'm a tutor or a resource instructor.

[chervilant]

149. Oh, noes!!!!!

You TUTOR?!?!

I hope you are better with your students than you are with your comments.

[Recursion]

150. *shrug* I think my description of the formula was enlightening to some on this thread

Though I wouldn't describe it the way I did upthread to a high-schooler (the third-order system would, though, a great way to get at matrices -- "So the teacher, school, and district all do things on their own, and all interact with each other. How many terms is that?" etc.).

[chervilant]

153. Well,

as I was reading those comments, I wondered if you've read 'Chaos Theory' by James Gleick. This tome (and Hofstadter's 'Godel, Escher and Bach') helped me recognize that we humans seem hellbent on imposing order on our chaotic universe, which some are coming to realize is futile and unnecessary.

As a gradual student, I took a preponderance of statistics and methodology courses, primarily because I suspected that such disciplines could be used unscrupulously. Turns out, I was right--much to my dismay. I see this 'measurement' formula hereinabove (a most ridiculous algorithm) as a fine example of such an unscrupulous tool.

[Recursion]

155. I love Gleick

But then again my educational background is engineering, not science ("Electrical engineering: the chaos is built-in"). Even fantastically chaotic systems are linearizable piecewise, though (that's all Newton's laws of motion and Maxwell's equations are).

To me the bigger question here is orthogonality: are the I's really linear coefficients? Would the same teacher have had a different VAM in a different school? (If they are orthogonal, by definition the answer is "no", but actual data rarely achieve true orthogonality -- note that that doesn't mean the model claims a teacher would not have seen bigger performance gains in another school, just that the calculated VAM would have to be the same).

As a gradual student, I took a preponderance of statistics and methodology courses, primarily because I suspected that such disciplines could be used unscrupulously.

Well, sure, which is why engineers tend to roll their eyes when stats people talk -- stochastic processes don't work like humanities majors often seem to think they do. However, from the look of it this model in particular doesn't seem horrible. And if the result here is that teachers will feel an incentive to work in PS 932 rather than the uber-awesome magnet school, well, that's part of the point.

I see this 'measurement' formula hereinabove (a most ridiculous algorithm) as a fine example of such an unscrupulous tool.

At the risk of being pedantic, it's not an algorithm but a generating equation. What seems unscrupulous about it to you? Teachers are rewarded if their students improve in a manner uncorrelated to the improvement of other students in the school and the district at large.

Now, why are we laying off teachers in the middle of a recession? That's a good question to ask.

[chervilant]

159. Ah, yes...

linears are such nice, easy organizational tools...

(BTW, I was being slightly sarcastic when I used the term 'algorithm' and my students would warn you that I use bad math puns all the time.)

[Recursion]

160. That was my population size complaint

Linear models are great and usually work reasonably well when applied to a large enough population -- I don't know of any real evidence that a school district is large enough. But I keep coming back to the idea that if teachers had come up with an assessment system that didn't end up with 95% of DCPS teachers being in the highest possible category ("distribution? what's that?") we wouldn't have to be having this conversation.

And math puns are great. I keep trying to work "algorithm" into something about Al Gore when he danced the Macarena, but it hasn't clicked yet. (I also like the fact that for centuries it was "Algorism", but people wanted it to sound like "logarithm". And "logarithm" is usually my favorite Ah-ha moment when a student gets it.)

[chervilant]

161. And,

we seem to be belaboring this point, which so richly illustrates both of our assertions about assessements hereinabove. The bottom line? Our system of public education SHOULD be vastly improved. Please, let's commend our energy to this effort.

(Logarithms are the COOLEST, and my students lovingly call me a math nerd when I make such statements in class.)

[LWolf]

22. Another point:

My district, as I mentioned awhile back, has to take on VAM because we took a TIF grant. So we've had Mathematica, a "non-partisan" private policy research group, educating us about VAM. One of the things that they do is develop VAMs for schools and districts. Another thing that they do? They "consult" for the federal government, and report back on whether or not a district's efforts to use student achievement to evaluate and pay teachers is "significant" enough to continue receiving TIF money.

They told us that there are only 3 companies in the nation qualified to build a VAM for any district. It's too esoteric for educators to understand, educators can't build their own, and the cost to build a custom system requested by a district is too high.

Effectively, districts must spend a chunk of that TIF money paying one of those very few companies to create their VAM, or lose the grant. There aren't enough groups creating VAMS to get a customized version, so when you pay them, you get what they've already decided on, fit to your district's demographics.

Why so few? Because the concept of VAM is brand new. UNPROVEN.

[Recursion]

63. VAM has been around for 20 years in education

And for about 40 years in some other disciplines.

[chervilant]

147. OIC...

You are copacetic with Value-Added Modeling (VAM), but NOT with assessments to measure students' achievements?

I get the impression you're just here to be argumentative, or divisive.

[Recursion]

148. I'm not terribly copacetic with either

But I don't have a better replacement for either. I brought up assessments because I think it's hypocritical for a profession that spends a lot of its time assigning more or less arbitrary grades to students, which will have a tremendous impact on those students' lives, to then freak out when somebody does it to them.

[chervilant]

151. Well, then,

you might want to cultivate a level of trust with some of your veteran teachers so that you can find out the subversive ways that they actually TEACH, despite our stultifying and pedantic system of public education.

Which leads me full circle back to my original adjuration: PLEASE research this issue further, and commit your energy to improving our system of public education! Our children deserve our advocacy.

[chervilant]

156. By the way,

I've posted this before on other threads, but it bears repeating:

our system of public education has been transformed into a corporate tool. By the time we hoi polloi get our high school diplomas, more than three-fourths of us are convinced that we have 'average or below average' intellects. What an enormous crock of bovine feces!

Each and every one of us has a fully functioning, amazing brain. Contemporary research on IQ tests suggests that most of us would score 'near genius' level on these tests--if the time element of the test was removed. I contend that each of us learns in our OWN way, at our OWN pace. If we were to revamp our system of public education to honor and celebrate this fact, profound things would ensue.

If you haven't already guessed, this is how I teach. My students struggle for the first few sessions to accept that they're brilliant, then they get right into that groove. It's AWESOME to watch this process!

[Recursion]

157. Hoi Polloi

more than three-fourths of us are convinced that we have 'average or below average' intellects

Umm.. if the distribution of "intellects" is Gaussian, wouldn't that be true? But I'm not personally convinced intellect is Eudoxian, let alone that IQ measures anything meaningful.

I contend that each of us learns in our OWN way, at our OWN pace. If we were to revamp our system of public education to honor and celebrate this fact, profound things would ensue.

Yup, they sure would. which is why I like tutoring.

[LWolf]

179. Your impression is wildly misinformed.

I'm not sure what I said to give you that idea; feel free to browse through my journal, including archives; most are about education and you can get a better picture of what I'm copacetic with, and not, there.

VAM? I'm an aggressive opponent.

Assessment? I like reliable, valid, appropriate assessment that informs instruction.

I oppose one-size-fits all standardized tests as a single measure, I oppose high-stakes testing, and I oppose the myriad misuses and abuses of assessment currently rampant in the system.

[Recursion]

184. He was talking to me, I think (nt)

[LWolf]

189. I guess I should put on my glasses when I get to the 'puter, lol. nt

[Recursion]

193. The danger of large threads (nt)

[LWolf]

177. I've been "around" in education for 28 years now.

Value Added Measures as formulas applied to students' standardized test scores as a way to evaluate and pay teachers have certainly not been in play for 20 years.

I can't speak for other disciplines, but I can speak for mine. I don't have to, though. Mathematica, the people chosen by the feds to oversee compliance with the TIF grant, using VAM, has already said that it's so new that there are only a few players in the field.

Here they are:

http://www.mathematica-mpr.com/index.asp

[Recursion]

185. I'd say that too if I were going for a contract

But that wouldn't make it true. Hanushek's paper was in the 1970's, and the first implementations were early in the Clinton administration in the south (93 I think, but I can dig it up if you need to know).

[LWolf]

188. Dig it up.

I can guarantee you that there has been no mention, let alone actual consideration for implementation, of what is currently on tap as "VAM" in the districts, large and small, and the states, across 1200 miles that I have worked in, or in national educational reports I've accessed, of a VAM using student test scores until recently.

[Recursion]

192. Links to the abstracts since I don't have JSTOR on my work computer

(Why, oh why, aren't all journals at DOAJ yet?)

Here's the first paper that addresses VA/6sig in education that I've heard of: http://www.jstor.org/pss/145575

It's from 1979.

The first implementation (that I know of) is described in Sanders, Horn 1994: http://www.springerlink.com/content/l801g2688u4k848k/
And apparently in Sanders, Saxon 1994, but Google Scholar is failing me on that.

As far as I can tell it was limited to Tennessee and North Carolina until the turn of the millenium or so, so your point about it not being done nationwide until recently is well-taken.

I totally am down with the idea that simple attainment of students is a bad measure to judge teachers by. But development of students should not be. And if the assessments we're giving to students are flawed (and they certainly are in some ways), I'm much more concerned about the impact of the tests on the students' lives than on the teachers'.

But, if student learning can be more or less accurately assessed, then you're stuck with a dilemma:

1) If some teachers are better at making the students improve, we should find out who they are (by, say, actually measuring the students' improvement)

2) If that's not the case, if no teachers can reliably increase the performance of students, then it doesn't matter who is teaching.

[LWolf]

197. Thanks for the links.

I have a more in-depth response, but am taking off in a few minutes for the reservation north of me; bringing local native history to teachers and students.

I'll hit it again this weekend.

[ieoeja]

79. More likely they are just big contributors.

When I was at the Veteran's Administration in '85-'86, I worked on a debt collection system. The final three steps of collection were:

1. refer to one specific credit reporting agency,
2. garnish tax refunds,
3. give 50% of garnished refunds to credit reporting agency.

Under normal circumstances, if someone pays up after being referred to an agency, you don't know if they finally got around to paying up, or if efforts of the agency caused them to do so. In this case, we obviously knew the garnished refunds had nothing to do with the CRA. But that is how things were setup.

Fortunately, the IRS is a whole lot easier to understand than the CRA. So I never got communications working with the CRA.

A shame, that. But it did save the taxpayer $7 million that year. And probably pissed off some large contributor wondering why he never saw any of that money the congressman promised him.

[thanks_imjustlurking]

101. Bingo - it's a money-maker for somebody

I've been saying that, also, about our state's customized standardized tests for years. Cash cow for the company that develops them.

Then the PTB in state government adjust the passing score and various other little formulas to make things look good or bad at the proper political times. Some system. Costs a bunch of money, takes up massive amounts of class time, and can be manipulated easily.

[LWolf]

181. Yes. nt

[Starry Messenger]

23. Prenology charts used to be considered science too.

So did palmistry. This whole VAM thing is total pseudo-scientific garbage.

[roody]

24. Somebody is getting rich off this garbage.

[madfloridian]

44. I could make a whole list of those getting rich.

And it is not the teachers. And the students are being truly left behind while the reformers are at the corporate trough getting fat.

[Blue_Tires]

+1

[Blue_Tires]

91. +1

[roody]

104. I know you are busy, but that would make a great thread.

[PVnRT]

28. That formula is nonsense

What in there is actually "measured"? It looks to me like a bunch of values that are ultimately subjective. I suppose some consultant earned plenty of money to come up with that.

[Recursion]

46. It's not that mysterious

Y_0, Y_u, and X_i are measured. Lambda and zeta are heuristic values from the model. The three alphas are found by solving a system of linear equations.

[AngryAmish]

54. In what way is it nonsense?

It measures improvement in knowledge and holds certain variables constant. If you have a bunch of dim kids you don't expect them to advance as much as if you have a bunch of smart kids. Teachers are not penalized for that.

[madfloridian]

61. Yes, teachers ARE penalized for that.

The new rules under NCLB are that there are NO excuses.

[Recursion]

62. Look at the formula; that's the opposite of what happens

The people who are penalized with this formula are people like this teacher who teach kids who are already performing well.

[JackRiddler]

76. Some of the quants who helped the bankers fuck the world were laid off and ended up in Ed.

[thanks_imjustlurking]

102. Bingo.

Besides, even if it is a common formula in business a) as has been said, kids are not widgets and b) it depends in large part on subjective data or data that can be manipulated easily, it seems to me from what I know about it so far.

[Zing Zing Zingbah]

187. I agree that using this formula for teacher evaluations is nonsense

The observations done by administrators and other teachers are the right tools to evaluate teachers. These are subjective too, but this is why several observations are done... to get a more accurate perspective of the teacher. If most people say the woman is a good teacher, and the formula doesn't, then I don't think the formula isn't the right tool to use. Isn't that just common sense? Even as parents, we can recognize some one who is a good teacher to our children. We don't need some frigging formula to figure that out... we never did in the past. Why would we now? Teachers get paid to little to put up this BS. This only serves to weaken our education system, not strengthen it, because this kind of thing is going to drive away even more of our good teachers.

[Recursion]

191. Or

If most people say the woman is a good teacher, and the formula doesn't, then I don't think the formula isn't the right tool to use.

Or, the school power structure is wrong about what a good teacher is.

Or, the point of the system is to push teachers into going to PS 932 rather than a closed-enrollment magnet.

Or, like in any other teaching evaluation system, she got screwed (are you telling me supervisor/peer based evaluation systems have never been unfairly harsh towards teachers the evaluators just don't like for whatever reason? Or unfairly kind towards the teachers they do?) and that's what happens when you have systems, from time to time.

Certainly it will suck if she gets laid off, but some other system would have made somebody else be laid off -- the problem here is that we're laying off teachers when we shouldn't be.

[xchrom]

29. recommend

[Recursion]

34. Why is the assumption that the metric is wrong, rather than people's opinion of her?

The whole problem is that teaching is notoriously difficult to assess and in a lot of places we aren't assessing it well at all. Why is it that surprising that objective criteria don't always line up with popularity?

[PVnRT]

36. Please point out the "objective criteria" in the equation they used

[Recursion]

37. Does "objective" have some new meaning?

Those are all defined by whatever instrument they use for them.

[Pacifist Patriot]

38. Because the opinions of her are based on both intangibles and

quantifiables. Read through all of the responses and you'll see some great examples as to why the so-called objective criteria is not remotely lining up with her supervisor's assessment of her performance.

Teaching is notoriously difficult to assess, but no one is suggesting assessments be thrown out completely. The problem is with this model. It fails miserably.

[Recursion]

39. But why do you say that?

The argument seems to be "obviously it's wrong because people like her as a teacher". Well, what if we like people who aren't good teachers?

[Pacifist Patriot]

50. You and I are not seeing the same argument then.

The argument I am seeing is that the model is flawed because it does not reflect reality. Her students have high test scores, decent grades and she is highly evaluated by her superiors. Others have pointed out other weaknesses in a supposedly objective formula that really doesn't pass the sniff test.

This has nothing to do with liking or not liking one particular teacher. She is being used as an illustration of how the model can have real life implications for individuals. They could just as easily have used a crap teacher who scored highly, but I doubt that would have the same emotional impact.

I'm not sure where you're getting the whole likability thing.

[Recursion]

59. Several things.

The argument I am seeing is that the model is flawed because it does not reflect reality.

Hm. It's based on the test scores that actually were made.

Her students have high test scores, decent grades

And we're told over and over again that that isn't teachers' responsibility: rich kids do well and poor kids don't.

she is highly evaluated by her superiors

Right: that's what I meant. She's popular, so this assessment is assumed to be wrong (it can't be because of the students' achievements, since again we're repeatedly told teachers shouldn't be praised or blamed for that).

Basically, she got hurt in this model because she taught the "easy" kids -- ones who are already performing well. That fact doesn't bother me all that much.

[Nevernose]

200. Because we have 20 years worth of data and studies

All of which amply demonstrate that, when measuring teacher efficacy, the VAM statistical model is not accurate. Despite myriad promises, VAM does not account for the SES of students, the particular test used, or other measures of success (e.g. graduation rate, college acceptance, etc.).

There are a lot of studies showing it doesn't work, no studies that it does (in a real world school setting). It is therefore safe to assume that it's the test that's the problem, and not the teacher.

[4lbs]

35. Even though I'm a statistics and numbers guy by nature (math, science, computers), I have a huge

problem with that statistical formula and how it is being used.

It relies so heavily on other arbitrarily derived values that if any one of them were improperly formulated themselves, the entire function is erroneous and worthless.

It goes back to the old GIGO term I learned decades ago.

GIGO = Garbage In, Garbage Out.

If your data input is invalid, erroneous, and/or meaningless, you cannot expect the result to be better.

I would like to know how all the other numbers in that impressive looking formula are calculated. How is the number "Student Characteristics" computed? What is each of the "Effect" and "Indicators" functions? "Student Error Term", what is that and how do they calculate it?

Furthermore, it may very well be that the formula itself needs correction to match real world results. How well tested is this formula? I have no idea. Are there any publications that state this is verified to match real world results?

If I create a formula that is supposed to predict or compute value or ranking, and it gets numbers that don't match common sense or what humans actually perceive the value or ranking to be, then guess what? I modify my formula until it closely matches what the results should be.

As an analog, let us say I examine migratory bird patterns and create a statistical formula to determine where a species of bird should be located given a certain day of the year. I examine a whole series of data and sightings to aid me in my effort. I come up with a formula that says that they should be near a certain geographic point on say, October 10. If that is incorrect, and is incorrect for other days (September 21, November 3, etc.), I don't say, "well, the birds or sightings must be wrong because the function says they should be here". It is my formula that needs adjustment. Either I missed something so that it is frequently invalid, or some other environmental factor has attached itself that was not taken into consideration.

Without a litany of tests and verification, this statistical ranking formula for teacher scores should not be the sole or even a major reference factor on computing a person's ranking, eligibility, and worth. This has real-world impact on many lives.

[Pacifist Patriot]

40. Recursion needs to read your excellent response.

[Recursion]

42. Why? The model looks like bureaucratic nonsense to me

I'm just bothered that nobody seems to be coming up with a better reason we should trust the assessments that it contradicts.

[Pacifist Patriot]

51. What better reason do you need besides it doesn't reflect reality?

The whole thing looks to me like a "baffle them with bullshit" formula so someone can get a contract with the municipal government and make a few bucks.

[Recursion]

53. I guess I just don't see what's so "baffling" about that formula

It looks pretty clear to me. It's what pretty much every systems analysis I've ever seen looks like.

[Angry Dragon]

86. +1000

[thanks_imjustlurking]

103. Thank you!

I meant to put GIGO in one of my responses, but my train of thought got derailed.....

[DirkGently]

43. Horrendous example of corporate blockhead logic applied to defile a higher category of endeavor.

What exactly will it take for all of us to reject this ridiculous dum-dum business world gibberish being used to bash teachers outright? They. Don't. Get it. You can't quantify teaching, and you mostly can't quantify learning.

This is largely why education, along with healthcare, cannot be run as a business proposition. There's not widgets to churn out as cheaply as possible, no marketing to create reality by perception. You need dedicated people who CARE about PEOPLE, which is antithetical to the profit model.

This notion that business people know best about everything has become a religion in our country, and a bad religion to be sure.

Money guys / women know how to make money. That's IT. You don't want them teaching your kids or caring for you when you're sick, or putting out your fires or watching your prisoners.

These things take levels of intelligence and compassion that no business model -- certainly no business model we'd call American capitalsim -- can handle.

It's beyond them. Put down the chalk and the overhead projectors, and slowly BACK THE HELL AWAY FROM OUR SCHOOLS.

Please.

[progressoid]

49. This just pisses me off.

:grr:

[Initech]

81. Me too. It's just more corporate bullshit logic that is used to support what they want.

Every day, the more I see it, I really do believe George Carlin's theories about the owners of this country are becoming true.

[Maraya1969]

95. This is a joke right? You can't have unable to prove variables and empirical

data make up a real math equation.

Seriously is this a joke?

[JHB]

105. This isn't a "scientific" calclation, it's numerology

It's the worst kind of joke: the kind that isn't funny, is in fact highly insulting, but the jackasses who control you livelihood think it's so falling-down funny they insist on telling it every chance they get and more.

[Recursion]

116. I just don't see what's so mysterious about it

There are linear systems like that all over systems analysis. Hell, the formula itself would be a good lesson for a high school linear algebra class. It's a well-known model for improving feedback-based systems.

[JHB]

142. It's not the formula itself that is so mysterious

It's an extreme (but justified) lack of trust on how it will be used, and the weight give to it rather than other assessment methods just because it provides a nice, tidy, "objective" number.

[Recursion]

144. Well, I don't trust peer and supervisor assessments all that much.

When I worked at DCPS, 95% of teachers were rated in the highest possible category, and a lot of the 5% who weren't, sued.

Even in less corrupt systems, there's always the problem that "watching them teach" is based on a prior model of what a teacher "should" do, rather than whether or not the kids are actually learning something.

[Recursion]

119. The Y and X variables are from tests and assessments

The lambda, zeta, and epsilon variables, and I coefficients, are priors from the model (presumably created from a data regression over a long period, but given the amount of BS that gets thrown around in education "reform", I may be too optimistic here). The alphas (or possibly lower-case a's, I can't tell from their font) are found by solving the resulting third-order linear system. The only "squick" factor is in the I coefficients, and those are set up long beforehand -- the epsilon, zeta, and lambda simply determine the size of the pie, not how it's being divided.

[Keith Bee]

106. Can't R anymore

But can K!

[callous taoboy]

108. I teach third grade and just finished giving the math test yesterday: Listen up:

Despite pouring every ounce of energy into my job, despite not being out once this year, and despite giving my three lowest math students hours of extra individual tutoring attention, those same three students yesterday finished their tests in one hour.
One fucking hour! As they checked their bubbling in front of me I could not help but notice that they showed absolutely no work on their tests. In other words, they performed exactly as they have all year: abysmally. They likely failed the test. My top math students, on the other hand, worked all day on the test and they had notes written all over them. They did their work.

WHY IN THE HELL should I ever be penalized for the work of students that either just don't get the importance placed on these tests, or they just don't give a fiddler's fart how they do? WHY?

[callous taoboy]

109. The people on this board who defend the crap the teacher in the OP is putting up with:

I want you to read what I wrote above and tell me what is fair about what I describe?

[Recursion]

113. Delete

Post error bug...

[Recursion]

114. OK, here goes

WHY IN THE HELL should I ever be penalized for the work of students that either just don't get the importance placed on these tests, or they just don't give a fiddler's fart how they do? WHY?

This is the fundamental problem, right? And it's not remotely reserved to teaching: in most jobs you are held accountable for things outside of your control. We don't have publicly-paid teachers so that they can have jobs, we have them so that children can be educated. It is argued, forcefully, that teachers need job security to educate children well, and I'm inclined to agree, which is why I'm not against tenure for primary and secondary educators despite the fact that the original concept isn't really relevant to what they do (tenure is meant to protect academic freedom for original research and publication, which is not something primary and secondary educators do a lot of).

Testing is too high-stakes; I certainly grant that. But from the standpoint of the children and their parents (and anybody else paying property taxes), if your students didn't care about the test -- and presumably the subject? -- then they need to find a teacher who can make them care.

"But wait!" some will say. "No teacher can overcome the poverty and awful family life these children bring to school with them. You're waiting for a Superman that doesn't exist." OK, that's possible. But if that's the case, why does it matter who's teaching them? If there's not some other teacher that could make them capable of demonstrating competence in the subject, why does it really matter who's teaching them?

[xphile]

126. Get a teacher to make them care? If it were possible to force someone to care

Then someone would certainly had managed to find a way to make you care about the plight of a human being who is being screwed by some seemingly impressive formula which is probably riddled with false assumptions disguised as variables and coefficients in some ridiculous pseudo-formula designed to fuck teachers over since they've managed to thoroughly turn the private sector into a cesspool where arbitrary firings based on bullshit is commonplace.

But you don't give a shit and no one can make you. Yet you seem to think that a child who doesn't care and isn't getting any incentive at home to care can be made to care by someone who is trying to teach a large classroom. Shall the teacher sacrifice the ability for the other students to learn in order to constantly go back to the same students (who in this example were getting extra time from the teacher in an attempt to help them catch up) who still don't give a shit after all this time?

Make them care? We can't get the people who run the Democratic party to care about teachers, about GOOD teaching methods, about treating people with humanity and we damn sure can't get Republicans to care about anything but money but you toss the line about getting someone to make these students care as though you aren't at all aware of the idiocy of your statement.

But clearly I'm wasting my time with someone who is more comfortable with numbers than actual human beings. So I'll make one last statement and I'll make it very simple so even your formula obsessed brain can get it.

People are not fucking widgets. You can't just drop a bunch of people in a mold and expect identical results as though people were molten metal or plastic being made into widgets. Children are not cars. There is no formula that works for every person and any these so-called reforms do exactly that.

If we really wanted to improve our schools we would be looking at what other countries do and try to emulate that. Smaller classrooms with teachers who are paid more and are respected rather than trying to turn the teaching profession into a assembly line type job where there's no thinking required.

[Recursion]

127. That was the point of my last paragraph

Then someone would certainly had managed to find a way to make you care about the plight of a human being who is being screwed

What "plight"? What "screwed"? She still has a job at a very good school.

Shall the teacher sacrifice the ability for the other students to learn in order to constantly go back to the same students (who in this example were getting extra time from the teacher in an attempt to help them catch up) who still don't give a shit after all this time?

Well, from my time as a student I mostly saw teachers being nice to the rich & popular kids and not giving a shit about the rest of us. But then again I wasn't in an NYC magnet that required a test to get into; I was in a small town Mississippi public school.

And, like I said, if teachers can't reach the students who aren't being reached, why do we really care who's teaching? The "good" kids will do fine no matter who's there.

People are not fucking widgets. You can't just drop a bunch of people in a mold and expect identical results as though people were molten metal or plastic being made into widgets. Children are not cars. There is no formula that works for every person and any these so-called reforms do exactly that.

And systems analysis, despite all of the BS that gets flung around about it, doesn't pretend that people are widgets or all alike -- in fact, it's based on the idea of variance over a population.

If we really wanted to improve our schools

I've yet to see any data that actually show public schools are in some kind of trouble that requires "improving" them.

[xphile]

128. The whole premise of this idiotic formula you seem to be so entralled with

is that the schools aren't doing enough to for the students and that's why we as a nation are falling behind the other schools.

You're being extremely disingenuous by pretending not to know what the whole argument behind the use of these type of formulae.

When you liken children to vehicles as you did earlier you are pretending that people are like widgets.

And your experience as a student doesn't translate to all teachers perhaps you should put your broad brush away.

And while I'm at it magnet schools ARE public schools.

[Recursion]

130. I never likened children to vehicles.

is that the schools aren't doing enough to for the students and that's why we as a nation are falling behind the other schools.

Is it? I thought it was that we don't really have enough data on what schools and teachers actually do and don't do.

You're being extremely disingenuous by pretending not to know what the whole argument behind the use of these type of formulae.

The argument behind these type of formulae is that it is possible over large populations to isolate to some extent what results come from what participants. And I'm not sure why you think I'm "enthralled" with them; I've criticized them in this thread, and specifically pointed out that they work better in populations much larger than a school district. I've also pointed out that peer- and supervisor-based assessments, which inevitably fall victim to old boys' networks, are not a priori better.

When you liken children to vehicles as you did earlier you are pretending that people are like widgets.

That wasn't me, and like I said you're mischaracterizing what systems analysis does.

And while I'm at it magnet schools ARE public schools.

I'm aware of that. So are charters. I was at a traditional public school. She could teach at a traditional public school, and would probably have done better by this formula if she had. Just guessing from the formula that seems to be one of the incentives it produces.

[xphile]

136. My mistake. It was your cohort who likened the children to vehicles.

But I stand by my comment about what you claim systems analysis does. You can't just plug people into a formula and think that's good enough to determine whether or not they are doing a good job when you do you act as though they are all alike and that is wrong. You seem to be arguing that the formula itself is a better indicator of who is a good teacher while saying that if the teacher wants their efforts to be acknowledged that they need to work at a shitty school. The point is to see how well the teacher teaches. Consistent excellence should not be penalized which is what is going on in this case because if the previous year the students do well there isn't much improvement to be done. This formula seems to be saying that if you're consistently excellent you're not good enough to stick around as a teacher. That is a ridiculous conclusion to come to and if this formula is coming to this conclusion then the problem is the formula not those who argue that it's a foolish one.

BTW, charter schools work outside the public school system while taking public school money. Magnet schools, which have been around long before this charter school craze, work within the public school system.

[Recursion]

137. Nobody's "plugging people into a formula"

You can't just plug people into a formula and think that's good enough to determine whether or not they are doing a good job

Nobody's doing that. The formula is a way of estimating what portion of the measured improvement comes from the teacher's work rather than other factors. And of course it's flawed, but so is any other system, and any system is going to wind up being unfair to someone (and I still don't see how she's being "screwed" or in "plight": she's still teaching at an excellent magnet school).

when you do you act as though they are all alike

That doesn't even make sense to me -- the whole thing is based on looking at the ways different students and teachers are measurably different, not claiming they're all alike. Maybe your problem is the fact that it's being measured at all, but then like I said above teachers seem to have no problem setting much of the course of kids' future by how the kids' progress is measured.

You seem to be arguing that the formula itself is a better indicator of who is a good teacher

I haven't particularly said it's "better" than anything, though I've criticized assessments based on a priori education theory models that then ignore whether the students actually learn anything (which, yes, is something of a parody, but not all that much of one).

saying that if the teacher wants their efforts to be acknowledged that they need to work at a shitty school

Well, that seems like the sort of thing we want to reward, yes?

Consistent excellence should not be penalized which is what is going on in this case

How do you figure? I didn't see any penalty for the teacher. She still has her job at the same pay.

This formula seems to be saying that if you're consistently excellent you're not good enough to stick around as a teacher.

Nobody's being fired here. The formula is saying that if you're consistently teaching already "excellent" students, your rewards will be less than those for a teacher who is teaching "worse" students regularly. And again I'm not even saying that's a great system, just that it's not on its face stupid, and I still don't see how it's worse than having your supervisors and peers evaluate you according to some prior model of what a teacher should do.

[SlimJimmy]

166. Wrong. I didn't liken children to vehicles. I said that LSS is system that

uses value add, and that the car they drove was very likely designed, improved and produced using that system. I said that in response to the allegation that value add was "baloney." Feel free to read any of my replies and QUOTE where I said children were like vehicles.

[callous taoboy]

164. I appreciate the passion with which you argue with this ill-informed

fool. Again, my thanks.

I have him on ignore. All teacher-bashers get the boot from my DU.

[callous taoboy]

163. "...then they need to find a teacher who can make them care."

Thanks, thanks a lot for that. The assumptions you make about me as a teacher say more about you than myself. I've taught for 17 years, and for the past three years I've had 100% passing rates on the reading and math tests. Those three years brought me very challenging students to reach, one of whom was living in a storage unit. I helped provide that family with clothing and shelter, AND I managed to get the boy to at least pass the tests.

You really are not worthy of a response, but I just had to say what I said.

Now to the good old ignore button so I don't have to put up with ignorant statements like the one I quote above.

Ignorance is truly a disease of the mind.

[Reader Rabbit]

111. Measurements that Mislead

Science writer Jonah Lehrer has an excellent take on why coming to conclusions based solely on statistics and test scores in not a good idea:

The reason maximal measures are such bad predictors is rooted in what these tests don't measure. It turns out that many of the most important factors for life success are character traits, such as grit and self-control, and these can't be measured quickly.

Entire article here:

http://online.wsj.com/article/SB10001424052748704471904576230931647955902.html

Those that argue in favor of statistics and test scores as the only "objective" measure need to read Iain McGilchrist's The Master and His Emissary. The too great influence of the left-hemisphere perspective in Western culture leads to a greatly dehumanized society.

[proud2BlibKansan]

162. The important point that no one in this thread has mentioned:

Let's go back 12 years when NCLB was originally pulled out of George Bush's posterior. It was based on the 'Miracle in Houston' performed by Rod Paige. And we now know that the miracle was bullshit and that Paige twisted numbers and stats to make it appear he had found the magic answer to improve our schools. That's point #1.

Paige went on to become Sec of Ed, was in way over his head, called teaches terrorists and eventually forced to step down. He should be in jail but that's another thread entirely.

Point #2: teachers across the country knew that NCLB would turn our schools into testing havens where the only ones who profited were the people who wrote the tests. We lobbied, we petitioned, we begged. We were accused of not wanting to be held accountable.

Point #3: here we are 10 years into this nonsense piece of legislation and there are only a handful of people in the entire country who think NCLB has been good for our schools or our children. It is a pretty universally despised law, by both parties, and it's a given that it should be thrown out or drastically changed. In other words, the teachers were right.

Now we have this value added crap being thrown at us and once again we criticize it and are accused of wanting to maintain the status quo. And of course the implication is that the status quo sucks and teachers are evil and stupid for not rejecting it.

And on and on.

I'm sick of this crap. I'm tired of being accused of not wanting to be held accountable because my professional opinion no longer matters. I'm up to here with experts who think they have the answers because they were once in 3rd grade and their teacher sucked.

It's not the teachers who are the problem-it's the system. It's also communities who don't support their schools and business folks who see nothing but profit instead of children.

[callous taoboy]

165. + 1, once again Proud. Please read what I posted above

and then check out the comments. Ignorance is truly a disease of the mind.

[proud2BlibKansan]

174. ..

:hug:

[chervilant]

196. Thank you

for your efforts regarding the ignorant 'devil's advocate' who relentlessly keeps missing the forest for the trees.

[madfloridian]

180. Applause.

:applause:

[Zing Zing Zingbah]

186. Who came up with that BS statistical model? n/t

[Recursion]

194. Motorola, originally. Or I suppose Gauss, really originally

And for all the rhetoric about it treating everything as being the same, the entire theory is based on the insight that every member of a population (be it a population of humans or mobile phones) is different from every other, and those differences in large enough populations will follow a Gaussian distribution, even if the underlying stochastic model is not Gaussian. As long as the variance is of a Eudoxian quantity.

Now, is education measurable? In some fields, at some levels, certainly, in other fields, at other levels, certainly not. But, like I keep saying, we're more than happy to let test scores run the kids' lives (way before NCLB).