## Friday, March 07, 2014

### Getting Testy About the SAT - Wrongs of Passage

Though the author of the piece linked below was an English major, while I was a math guy in high school, I pretty much agree with her assessment of the SAT.  Let me add a bit from my perch.

On the test itself, the math side is less about memorization than the verbal portion of the exam.  There is reasoning through algebra and geometry problems, and the beloved word problems, each of which need to be translated into an algebra or geometry problem to obtain a solution.  I wish that had been the entire test when I took it, but for some reason the authors felt it necessary to have one or two questions about Roman numerals.  I know them through 100 and understand the schema for representation, but confuse M and D, always have.  It seemed to me silly to have a question about that then and I haven't changed my mind about it since.  Then there were also questions of the type: 1, 4, 9, 16, what is the next number in the sequence?  These are pattern recognition questions.  There is an intellectual issue with these sort of questions - more than one pattern can be found to explain the sequence so far.  In the above, for example, the pattern might be odd, even, odd, even, with the numbers increasing, rather than each number having to be a perfect square (the latter being what the SAT would say is the "right" answer).  So the math test is not without criticism, but it's weakness are probably less severe than the weakness of the verbal exam.

On the consequences of the exam for the self-esteem of the student, the author appears entirely okay on grades in school, both as measure and motivator.  I am less okay on that as I believe it does quite a bit of damage to kids who don't rise above all that grading and figure out to learn for learning's sake.  (See my post College without Grades.)  Given that the grading system itself creates a lot of stress on the students, much needless in my view, what does the SAT contribute to make the problem worse?  Here, I believe the real issue happens when the grades as signal give a different message than what the SAT scores as signal provide.  In particular, students with a high GPA at a reasonably good school but who perform in a mediocre way on the SAT take a real blow to the ego.  No good comes of that.  That exam score, known to teachers and classmates alike, becomes that student's personal Mark of Cain.

There is then the issue of whether the SAT predicts college performance.  I will cite a few bits of different sorts of evidence.  First, for a couple of years I was on the Campus Honors Program Advisory Committee and one part of that was to review the applications of students to the program to determine who will get in.  Many of the applications I reviewed (not a great number in total with each an interest in economics or business) were from students who had perfect standardized test scores, whether SAT or ACT or both.  Students also had a variety of extra curricular activities which, to my non-discerning eyes, distinguished them from one another not at all.  What was left as a differentiator were the rather short essays.  (The U of I doesn't seem to require letters of recommendation or those weren't included in the packets we reviewed.)  I should add here that students don't apply for CHP - the applications are singled out based on other factors (the high standardized test scores and/or the geography of where the application is from).  So the kids may not have put in much effort into writing the essays.  Alternatively, insufficient guidelines were provided to inform them about what would please me as a reader.  With those caveats, I was totally underwhelmed by the writing, with very few exceptions to the rule.  So on the one hand you had these great test scores while on the other hand you had these mediocre essays.  What should be concluded from that?

Next, before I attended the Frye Leadership Institute in June 2003, we attendees were asked to read The Gatekeepers by Jacques Steinberg, which is an in depth look at the admission practices at Wesleyan University.  One gets the impression from reading that book that places like Wesleyan really do sort through a wide variety of information about each potential student.  Given that, perhaps the standardized test scores at such a place provide little value add to the overall picture of the student.  But what about places like Illinois, where the volume of applications has to be more than an order of magnitude greater than what it is at Wesleyan, and where such in depth looking at students doesn't happen.  (Most students who apply to Illinois do not have an interview with an admissions officer.)  Could Illinois have a reasonable admissions process if there were no standardized testing?

Then I'd like to talk specifically about performance on the math part of the test.  When I was a graduate student my adviser told me that the math part of the GRE (very similar to the math part of the SAT) was a good predictor of performance in Econ graduate school (which is very demanding about the math).  Much more recently, I was told something similar about the math score on the ACT as a predictor of performance in General Chemistry.  But my sense of things is that these predictions are not linear.  Outlier good performance on the standardized test (in my recollection of the scoring scale, here I mean above 700 as a score on the Math SAT) does predict quite well good performance in mathy-type courses.  But there is some threshold below which the standardized test performance doesn't predict well at all.  I say this based on my own teaching.  All the students I teach had to have reasonably good standardized test scores to get admitted to the university, but many of the students I've had over the years don't know analytic geometry and can't reason through fairly elementary deductions, even after the math facts have been explained to them.  Somehow there is a way to "fake it" on the standardized test, get a tolerably good score on the math part, yet remain largely math phobic.  If that is right, the scores of such students don't differentiate them from their counterparts, who also are uncomfortable with the math but score less well on the standardized tests.

Finally, there is the issue of wholesale cheating on the tests, particularly with international students.  For public universities like Illinois, this is a big deal as international students are apt to pay the full out-of-state tuition and thus provide a surplus to the university beyond the cost of their own education.  Revenue strapped public universities then might want to admit more such students, irrespective of how those students will perform once admitted, but they don't want to see a decline in ratings from doing do, hence maintain stern standardized test cutoffs for admission for international students, thereby providing a strong impetus for the wholesale cheating.  The entire situation is ethically murky, at best.

As long as there are highly selective universities, because a degree from one is perceived as a lifelong advantage, gaming of the system will be the norm.  The SAT is part of that system.  It is easy enough to criticize as an object in itself.  But maybe we should instead be talking about the entire system and ask what can be done for the system to function better and be more humane.  The author of the piece linked below is right that we shouldn't freak out the students.  The SAT does that.  And the rest of the system does it too.  We should be talking about it a lot more.