Better Screening for College Entry? Or No Screening at All?

College Admission : Harvard
Teen Health : __________
a) Acne,
b) In-grown toenail,
c) Obesity,
d) Manic depression,
e) All of the above.

I only recall one class in high school that with deliberation tried to prepare us for the various standardized tests which would follow.  This was English in ninth grade.  We did vocabulary exercises - new words to memorize, quizzes on their usage that followed. I don't remember whether we specifically had the dreaded verbal analogy questions or not.  I was not very good at those questions, nor was I good at remembering word meanings for words outside my working vocabulary. It's tough to get a verbal analogy question right if you don't know the meaning of the words.  Sometimes it's still tough, even when you do.

You might think it more fitting to have these type of drills in 11th grade, when the students take one standardized test after another.  Or possibly in 10th grade, in anticipation of all the examinations to follow.  Or you might think that back in the early 1970s there'd be none of this whatsoever.  The schools weren't graded by how the students did on the PSAT and the SAT.  So why coach the kids that way?

The explanation in my case was that my English teacher in ninth grade was also in charge of the College Placement Office.  All applications to colleges went through him and his secretary.  He internalized the benefit of placing students at elite colleges, even if the rest of the school didn't care.  So we got the vocabulary drill, whether it would actually help us two years later or not.  The far longer lived lesson, one I will never forget, is to detest verbal analogy questions.  This seems all the more remarkable in my case, since as my regular readers know I get genuine joy from making puns and doing other word play.  There is nothing like school to take the fun out of learning, by forcing it into a memorization schema, especially when the kid has his own way of learning already and is making progress with that.

As it turned out, I did apply to Harvard.  I got rejected, of course.  My verbal SAT score was too low.  But perhaps more telling in my case was the interview I had at the Harvard Club in Manhattan.  Wearing a suit, which I was not comfortable doing, and with the dark tones about the place from the wood paneling and padded leather chairs, I felt out of my element from the start.  I had the shakes and the sweaty palms.  It was a time when I'd say anything just to please the interviewer, prostitution without the sex.  Somewhere in the middle of the conversation, I recall telling him that I was a plodder.  Why I thought that would be a sell point I don't know.   Perhaps it was a way to show modesty.  But it was completely inaccurate.  I was then and still am highly intuitive in my thinking, doing most of it by how it feels, doing it quite quickly then, a bit slower now but still pretty quick much of the time.  Either for lack of self-awareness or because I was overwhelmed by the circumstance, I chucked the real me for some plastic imitation.

In the grand scheme of things this was all no-harm-no-foul.  I got a reasonably good education thereafter, though I had my struggles in college, which I will not belabor the reader with here as I've done that elsewhere.  The reason for bringing up the above at all is to set the stage.  The reader should ask herself what features of potential applicants would make them attractive to colleges?  How did the reader do on these metrics when she went through the process?  Is being an attractive applicant good for the student in the life to be led post graduation from college?

These questions were triggered by a piece Adam Grant wrote for the Sunday Week In Review, Throw Out the College Application System.  The piece leads off as follows.

THE college admissions system is broken. When students submit applications, colleges learn a great deal about their competence from grades and test scores, but remain in the dark about their creativity and character. Essays, recommendation letters and alumni interviews provide incomplete information about students’ values, social and emotional skills, and capacities for developing and discovering new ideas.

Adam Grant is something of a wunderkind academic.  I learned about him a year and a half ago reading this piece in the Times Magazine, Is Giving the Secret to Getting Ahead?   I liked that essay very much, even if the underlying theme was not novel to me.  Among other places, you can find it in Akerlof's paper about Labor Contracts as Partial Gift Exchange and Okun's piece The Invisible Handshake and the Inflationary Process.  It was good to see this theme about human decency in the workplace emerging again.  It was about time.  And Grant seemed a living emblem for the idea, walking the walk in his own way of interacting with students and colleagues.

So I approached the piece on College Applications with eager anticipation.  But my reaction to it was first disappointment, then anger, then wondering whether Grant should have a long conversation with Carol Dweck about what college is/should be about, and finally that this piece needs some critique from an economist like me.

The upshot of Grant's argument is that the wrong people are getting into the very good colleges.  These are spineless and unimaginative kids who nonetheless are excellent test takers.  Let's not admit these kids but instead take in the creative and high minded.  What could be wrong with that?  If you buy the core hypothesis then you'll likely buy the rest of the argument too, that there are ways the kids you want to admit can be identified through the right sort of psychological testing.  So let's move to a system with that sort of testing post haste.  (Grant is a psychologist and much of the piece is an assertion about the reliability of such testing.)

Grant ignores, however, that the kids who are currently getting through (and their families who are providing the encouragement to do so) are gaming the system and indeed that much of their lives has been spent jumping the current hurdle and focusing on how to get over the next one.  There was gaming of the system back when I was in high school, witness what my ninth grade English teacher did. But it seems pretty clear that the gaming is much more intensive now than it was then.  That observation, in itself, suggests several follow up questions that seem relevant but are absent from Grant's piece.

1)  Does it make sense to have a more effective screening mechanism for admission to college if the forces toward gaming the system remain unabated and might even be accelerating?  My sense is that, no it does not.  Indeed, without reducing the gaming itself, changing the way admissions are done amounts to a kind of gaming by the particular college.

2) Contrary to what Grant asserts in his piece, might it be possible for well trained gamers to credibly fake creativity and high character?  Here I am reminded that when I took the SAT the College Board asserted unequivocally that coaching via what we now call test prep would have no effect on student performance on the exam, because it measured aptitude.  Nobody believes this anymore and test prep is a booming business.  All the evidence that Grant cites, I suspect, come from measurements taken where the system has not yet been gamed.  Does that evidence speak at all to what might happen a la test prep if the approach to admissions were altered?

3)  What is the cause of the gaming itself?  Can that be alleviated to some degree?  Wouldn't our efforts be better placed in that direction than in perpetuating the current regime but screen better?  As many others have pointed out, a good bit of this is simply excess demand being manifest.  Population growth has increased the demand for slots at elite colleges (including the demand from international students).  The number of slots have not grown as fast, though at places like Illinois there are now roughly 20% more slots than when I started here back in 1980.  I don't know what those numbers look like at Harvard or other elite places, but I suspect the pipe has not grown fast enough.  The other obvious factor is the rising income inequality, which has increased the return to elite college degrees, even as the average return across all colleges may be falling.  Thinking this way, is better screening what we want or should we be arguing instead for more supply at the high end?

4)  Can bright but spineless kids be fundamentally transformed into responsible and creative individuals?  And shouldn't that be the purpose for college?  If you pre-screen for these attributes aren't you getting college off the hook regarding what it should be about?  I have to say here based on my own teaching that if the answer to these questions is yes, then that yes must be qualified with - but this will be very hard to do.  The students have been trained like Pavlov's dog to be hoop jumpers.  They've had many years of that sort of conditioning.  There is a lot of unlearning that needs to take place for those yes answers to emerge.

5)  Do the answers to these questions change when going from small scale (one institution) to large (the entire economy)?  I believe the answer to this is yes.  A couple of years ago I wrote a post, Gaming The System Versus Designing It.  While I considered issues other than college admission in that post, I believe the upshot is still applicable here.  We have become very good gamers, but we remain very poor designers.  We cling to a belief that what will work in the small can work in the large, all the while ignoring internal feedback loops that the will emerge and tend to counter achieving the results at scale.

But a design answer is what we need here.  The problem is both worthy and complex.  Let's not assume a quick and easy answer will do the trick.  It won't.