Headlines from several sources the other day announced that the folks who run the college SAT testing for high-schoolers were not only revising the tests significantly but even making the essay part optional. Well, I thought, this sounds like just one more step in dumbing-down the education system. I was wrong, and things aren’t always what they seem (he said tritely). There was much more to the story than the superficial treatment given by most sources. Most of those implies that public complaints as being the primary cause of the changes.
Educational testing is a big deal. Really big. It’s grown into a $4.5 Billion dollar industry that seeks to satisfy the demand to capture the academic potential of students and to sum that up in just a few numbers. It works poorly, so poorly in fact that a few colleges are already abandoning testing and relying only on school records and testimonials. Studies seeking to find correlation between test scores and success in college have been consistent in finding almost none. But, what about the essay? Surely a student’s native ability for self-expression is an essential element in any formula for success, so how can they abandon that? (Students who want to continue that tradition will still be able to, and colleges who want it can specify that they need it.)
A New York Times reporter investigated and reported on how the changes came about. It was enlightening. The new president of the College Board, one David Coleman, is a major force behind the changes. This is a guy who thinks outside the box and who has been a long-time critic of the testing system. In talking to an MIT professor, one Dr. Perelman, he found that the essay part of the test can be gamed. Here, from the NYT article, is how:
Since 2005, when the College Board added an essay to the SAT (raising the total possible score from 1,600 to 2,400), Perelman had been conducting research that highlighted what he believed were the inherent absurdities in how the essay questions were formulated and scored. His earliest findings showed that length, more than any other factor, correlated with a high score on the essay. More recently, Perelman coached 16 students who were retaking the test after having received mediocre scores on the essay section. He told them that details mattered but factual accuracy didn’t. “You can tell them the War of 1812 began in 1945,” he said. He encouraged them to sprinkle in little-used but fancy words like “plethora” or “myriad” and to use two or three preselected quotes from prominent figures like Franklin Delano Roosevelt, regardless of whether they were relevant to the question asked. Fifteen of his pupils scored higher than the 90th percentile on the essay when they retook the exam, he said.
Over the course of their two-hour conversation, Perelman told Coleman that he wasn’t opposed to an essay portion of the test, per se; he thought it was a good idea, if done well. But “when is there a situation in either college or life when you’re asked to write on demand about something you’ve never once thought about?” he asked. “I’ve never gotten an email from a boss saying: ‘Is failure necessary for success? Get back to me in 25 minutes?’ But that’s what the SAT does.” Perelman said that tutors commonly taught their students to create and memorize an all-purpose essay that contained the necessary elements for a top score — “a personal anecdote, a few historical references; Florence Nightingale seems a strangely popular reference.” When test day comes, they regurgitate what they’ve committed to memory, slightly reshaping depending on the question asked. But no one is actually learning anything about writing.
What the article does not say, explicitly anyway, is what flows naturally from this. The SAT people are using software to grade essays. It makes sense. Just consider the problem if they tried to hire English majors to do the job. They would have to give them something like these instructions.
We would like to hire you part-time for six months to do nothing but grade essays by high-school students. Since we have hundreds of thousands of these, through-put is important, so you will need to do this fairly and to be consistent with the output of the other 500 English majors we’ve hired of course. We expect you to complete at least 200 papers per week. Your results will of course be sampled by peer-review for quality.
Would you take that job? I wouldn’t, not unless I was starving and desperate. No, they’ve got to be doing it with software, hence Perleman’s analysis for tricking the system.
Self expression is vital, but I see no better way to gauge it than by lessening the testing burden on schools and freeing teachers to teach the way they should and not just “teach to the test”.
New York time reporter Todd Balf did a fine job with “The Story Behind the SAT Overhaul”. Those interested in education will, I think, find more to like at the link.