Another Problem with Standardized Tests

I loved this article. Todd Farley writes about the grading of standardized tests in an op-ed piece in the New York Times yesterday.

He points out the arbitrary nature of how writing samples are graded for standardized tests.

When I was a graduate student, I spent a couple of months grading MCAT essays. It was a horrible experience. (I imagine I signed a non-disclosure agreement. I hope it has expired!) What I discovered was just what Farley is talking about--scoring was, it seemed to me, completely arbitrary. I spent my summer trying to guess what score (1-6) the bosses would give the essays I was reading.

I could never discern exactly how to interpret their criteria. I usually could get close, but much of my summer was taken up by conferences trying to figure out why my scores were so far out of whack with others. I wonder if my students today would say that about my classes?

I suppose you could chalk it up to me being a 22 year old graduate student who didn't know what good writing was. You could also blame it on poor training by my supervisors. I don't know why I couldn't understand the rules, but I stuck it out because it paid pretty well.

It also gave me an immense amount of skepticism when it comes to looking at test scores. It made doubt their ability to tell us much about a person's competence.