Pith or Myth?

Danny Heitman's essay, Keep It Short, A NY Times Opinionator column, comes in at 1,038 words, not counting the author's name.  In other words, it is longer than the standard Op-Ed that appears in the print version of the paper.  Yesterday the Times ran an Op-Ed, Confronting Putin's Russia, that was considerably longer.  And Thomas Edsall's column is regularly quite long.  Does the Times itself believe Heitman's argument?  Or is it just posting it to play devil's advocate?

I write lengthy pieces with some frequency and may lose some readers by doing so.  Should I care?  I don't expect to make a dime from my writing.  Given that, I don't see why I should. 

Finding meaningful social innovation

My Facebook sidebar had a blurb which asks, "Are you a thought leader?"  My answer, "I wish I were, but apparently am not."  Take a look at the following paragraph.  It's from the paper The SCALE Efficiency Projects, published in JALN almost seventeen years ago.  The paper was co-written by the SCALE evaluation team and me.  I believe this paragraph was entirely mine.  In any event, ask yourself about labor-for-labor substitution in instruction, where the inexpensive labor is not adjuncts, but students and where the type of interaction envisioned is not one-on-many but rather one-on-one or one-one-a-few.  Nowadays, who is thinking about learning technology innovation this way? 

arvan.PDF - v2n2_arvan_1.pdf via kwout

A few years after The SCALE Efficiency Projects, Brown and Duguid came out with their book, The Social Life of Information, which got a lot of attention at the time. (Brown was one of the keynote speakers at an Educause national conference held in Indianapolis, where he gave a superlative talk.  This cemented the importance of what he was saying among my peers in the CIC Learning Technology Group.)  One might have thought The Social Life... would have amplified interest in the SCALE Efficiency projects and this idea of labor-for-labor substitution.  But it didn't. 

The old NLII idea of capital-for-labor substitution still dominates our thinking about learning technology and how efficiency can be attained from applying technology to instruction.  You certainly see it with the MOOC fascination.  But it is also there with classroom technology, particularly clickers. 

In the session with instructors talking about their use of clickers that I wrote about last week and that is linked in the previous paragraph, one universal reported by each of the panelists is that students don't want to look stupid so are reluctant to ask questions in class for that reason.  Clickers address that problem in the one-on-many setting of the large lecture class?  Given the high degree of awareness of the issue, one wonders why there hasn't been equivalent attention paid to encourage students to open up in an office hours setting.

In fairness, the folks who teach introductory physics at Illinois do seem to have a solution to this, but the approach hasn't transferred broadly.  One reason for this is that some of the large classes taught in other disciplines are very poorly resourced and the approach taken in physics doesn't transfer.

Let me try to address the above.  If undergraduates who have taken the course are used as peer mentors/tutors then they need to be paid and that might seem to be a cost add.  But businesses have figured out long ago that if they can make payment in a currency of their own choosing, it is much cheaper than paying in cash.  (Think of airlines who bump passengers willing to give up their seats.  The payment is in the form of a free flight to be taken at a later date, where there may very well be empty seats.)   In the use of undergraduates, a similar idea can be employed by noting that the skills entailed in being an effective peer mentor/tutor are essentially the same as the skills one needs to develop to become a good leader.  So by offering candidate peer/mentors some coursework on leadership leading to a leadership minor and having the peer mentoring itself be the practicum associated with that coursework, one can get this labor input at very low cost.  (The second or third time through as peer mentor, the student should be paid but then too, some of the work should change to mentoring the mentors.)

This sort of innovation is right there, in potential terms.  But the potential has not been exploited at scale.  (A few instructors embrace the approach but they don't seem to generate coattails.)  I can't say the MOOC phenomenon is crowding out this type of innovation.  MOOCs are fairly recent and this use of labor-for-labor substitution has been around for quite a while before MOOCs appeared on the scene.  Yet the idea of labor-for-labor substitution didn't capture the larger imagination within Higher Education.  I don't have a full explanation for why, but I do recall one colleague telling me in the 1990s that he couldn't use undergraduates in his very large class, because the idea wouldn't play well in the Chicago Tribune.

Last Friday Paul Krugman had a column called The Timidity Trap where he argued that the good guys among economic policy makers (those who wanted to see more fiscal stimulus) have been cowed into taking half measures only.  I believe something similar has happened in Higher Ed.  Our past seems to mark a willingness for change at the fringes with no change at all among the majority practice and to keep with that right up until the model is broken.  (The model at Illinois would be broken now if not for the influx of international students who pay full out-of-state tuition.)  The good guys in learning technology whom I knew and continue to interact with on occasion also seem content to pursue half measures.

And perhaps they wouldn't agree with this diagnosis that using undergraduates as peer mentors should be the focus of a new approach.  But if they do agree, maybe we should all take a lesson from Krugman and be less timid in arguing for this.  Now is time to turn up the heat. 

The broad side of a barn

For those of us who believe in the jinx (I'm one of those) the News-Gazette sports section had a very odd column yesterday morning, which featured quotes from the players about whether they'd prefer to be on vacation (it's spring break here) or practicing basketball because the season wasn't over yet.  While they said vacation is nice, all of them wanted to be still playing basketball.  Joseph Bertrand said he came to Illinois to play basketball.  That line surely was a gaffe.

"A gaffe is when a politician tells the truth" [Michael Kinsley]

Bertrand's line is innocent enough, but the politically correct answer would have included something about getting his education too.  Apparently, Bertrand was not on guard while speaking with the friendly sports reporter, Marcus Jackson

Now Bertrand and Jon Ekey, the two seniors on the team, and the rest of the Illini too can take a holiday.  They lost to Clemson in the second round of the NIT in a close game that was excruciating to watch.  The Illini outside shooting was terrible.  They were something like 0 for 12 from outside the 3-point line before they finally hit one.  That explains the loss and the title of my post as well.

Clemson is a good defensive team and they packed the lane to make it difficult to get layups.  We're a good defensive team too.  The upshot was not a lot of scoring, particularly in the first half.  On offense, we seem to shoot better from the outside and click more as team after some player has made a couple of layups.  In the last couple of games, that player has been Rayvonte Rice.  He finally figured out how to penetrate and avoid getting his shot blocked.  After that, Ekey started to make some 3's and we took a slight lead, but we couldn't hold it.

There is an issue in all of sports about how reproducible an action is - a free throw in basketball even more than a three pointer, a golf swing, the roll of a bowling ball, etc. - especially when there is a lot of pressure on.  Practice helps make the action autonomous and therefore reproducible.  Pressure makes it conscious again and then subject to momentary idiosyncrasy.  Malcolm Hill, who seemed to make 1 or 2 threes each game since he was inserted in the starting lineup, succumbed to the pressure.  Our other starting freshman, Kendrick Nunn, did the same in the previous game against Boston University.  One and done can do those sorts of things to a player, even a very good player. 

After they've had their well deserved break, I hope the players practice their outside shooting - a lot.  It will be hard to be a fan next year unless the shooting picks up.  Toughness is a good feature for a team to have and the Illini showed that.  But making outside shots on a regular basis would really help.  Grit with little offense is a formula for close but no cigar.

There are very few days where I watch more than one basketball game, but the first weekend of the NCAA tournament provides the exception.  I caught the tail end of Kansas against Stanford and then watched most of Kentucky against Wichita State, which had to be the best game yesterday.  It was played at a very high level, with both teams demonstrating intensity and proficiency.  One of my Facebook friends, Mike, posted something during to the effect that the refs were handing the game to Kentucky.  That may have been true.  Wichita State got no calls, including on a 3-pointer that went in off the bank, where the player claimed his shooting elbow had been bumped.  Of course, players always claim they are in the right and the replay showed very little if any contact.  But if there was contact and it had been a four-point play, the outcome probably would have gone the other way.

Perhaps more to the point, the Wichita point guard, the catalyst for the team, got into foul trouble.  So he was in and out of the game in the second half.   It meant that other players had to pick up the slack, and they did.  It's what made it such a great game.  For the most part, when the teams had open looks they made their shots.  And sometimes they made their shots even when the action seemed a bit forced. 

In the Big Ten Michigan has that shooting capability but possibly lacks some toughness.  It would be good to see teams with both.  Perhaps Michigan State is in that category.  They are my pick to win it all.  But for the league as a whole, if it is to be the top basketball conference, there needs to be two or three teams like that, in which case I'd like to see the Illini in the mix.

Except for Kentucky, the Tournament has not been kind to teams that feature a freshman who has been touted as a lottery pick in the NBA draft.  Since John Calipari has taken to feature playing freshman at Kentucky, it seems that other big name schools, Duke and Kansas in particular, have embraced the same approach, simply as a way to compete for talent in recruiting.  It seems to me to be unhealthy for the game. 

The last really good player to come out of Illinois was Deron Williams, drafted #3 after his junior year.  But coming out of high school, he was the second best player on his team.  Bracey Wright the bigger name at the time and then a much better outside shooter.  Players do develop in college, some more than others.  The big name ones out of high school have a lot of pressure on them up front.  Conceivably, that can impede their development.

I have no problem with a junior who is clearly very good and who likely will be a lottery pick declaring for the NBA draft.  By that point the player should have quite a good idea of how well he can compete against other college players.  But with freshmen, the hype and the reality can conflict and there needs to be patience to sort this out and let the player blossom against better competition.  There is also the matter of gaining physical strength and possibly adding weight (muscle) in the process.  That itself takes time.

So I'm happy that none of our current freshman are leaving and they have a chance to mature as players.  Here's to next year's team. 

Upset Specials

How many Cinderellas get to the dance?
Does Stephen F Austin have a real chance?

Shaka's charges played none too smart
Creating an opening for this upstart.

Dayton will be there and with no excuse
After dispensing with Syracuse.

They're flying high for yet another day
Making their free throws and with heady play.

Then there is Mercer who won by no fluke
In the round of sixty four beating Duke.

Next up for the Bears will be Tennessee
So another double digit seed will make the party.

At least one Cinderella and possibly three
Showing that with college hoops there's parity.

In the spirit of Chuck Berry the perfect scene
Will happen next week, Sweet Little Sixteen.

Clickers and Curiosity

On Wednesday I attended a Teaching with Technology Brownbag at the U of I.  The session featured a panel of experienced instructors talking about use of clickers in their (large) lecture classes.  The panel was moderated by Jamie Nelson of CITES.  The panelists were

• Darin Eastburn, Crop Sciences, Student perceptions of student response systems
• Lena Hann, Kinesiology and Community Health, Anonymous questions
• Brad Mehrtens, Molecular and Cellular Biology, Learning Catalytics – large enrollment course, group discussion
• Mats Selen, Physics, Co-inventor of i>clicker
• Julie Shapland, Accountancy, i>clicker in large enrollment courses

Ultimately, you can watch a recording of the session online.  The video hasn't been posted just yet.  It was a good and interesting session.  All the panelists make effective use of clickers in their classes to engage the students.  Four of the five use iClicker, which was invented here. Brad Mehrtens uses a different product, Learning Catalytics. 

I don't want to focus on the technology here, other than to note the obvious.  The clickers record the responses of each individual student (with iClicker this is the letter chosen in response to a multiple choice question, with Learning Catalytics more complex responses can be recorded) and then presents to the class the distribution of responses.   So clickers are unlike the traditional hand raising to ask a question, because with hand raising the student's identity is revealed while with clickers the identity associated with a particular response remains hidden to other students. 

Instead I want to begin with a very primitive question.  Why wouldn't the students be engaged in the class if there weren't clickers?  There seem to be three possible answers, two pertain to the course, the other to the students.  On the course, there is the subject matter (answer one) and where the lecture is held (answer two).  On the subject matter, Lena Hann's class is on human sexuality.  One might reasonably hypothesize that all students enrolled have inherent interest in the subject, even if the students are taking it to fill some requirement.  (I wish I had taken such a course as an undergrad.)  The other four courses I would describe as technical, meaning there is specific information delivered that requires specific skills or knowledge of prior context to process the information.  A lay person wouldn't have these specific skills or context specific knowledge.  Is there inherent interest in technical subject matter?  Let me leave that question for the moment, though it is the heart of the matter.  Before returning to it, I want to get at the other possible reasons for lack of engagement.

The second is the room where the lecture is held.  Foellinger Auditorium is the largest lecture hall on campus.  It can accommodate over 1,000 but campus policy caps large lectures at 750.  The next largest lecture hall is Lincoln Hall Theater.  According to the table on the FMS site, current capacity is 615.  There are then a handful of classrooms with capacity around 300.  In any of these large classrooms the space itself might distract the student.  Possible causes include - not being able to hear the instructor, not being able to see what is projected on the screen, physical discomfort from the seats, and classmates doing non-class things that are not detected by course staff and that serve to distract students sitting in close proximity.  Each of these issues is more pronounced in the back of the classroom.  Julie Shapland, who teaches in Foellinger, gave an interesting hypothetical about her walking right up to a student who was not participating with the clickers.  Were that to happen the student simply would get up and leave and she would have no idea who the student was, so she couldn't give a bad grade to the student for the behavior.  In effect, this means extrinsic motivation in such classrooms must be provided by carrots, not sticks. 

The third answer is that most millenials are manic multi-processors.  If they are not texting or checking Facebook every few minutes, they go bonkers.  Mats Selen, and I believe the other instructors who teach introductory physics as well, have a policy that students must put away laptops and cell phones during class so they aren't distracted by their electronic social tendencies.  Most instructors don't have such a policy and while Brad Mehrtens lauded Learning Catalytics, he also said he wasn't so naive as to believe students didn't check Facebook while in lecture.  One should also observe here the possibility of old fashioned diversions, e.g., daydreaming.  And it is also possible that a student who is paying attention nonetheless lingers on a point to make better sense of it, while the instructor has moved on.  On this last one, Darin Eastburn talked about using the iClicker to determine whether a significant segment of the class is confused on the current point, in which case the instructor should not move on but instead try to understand the source of the confusion and then rectify the matter as best as possible.

There seem to be two principal uses of clickers.  One is to gather attitudinal information, where members of the class are likely to vary in their held views.  The clicker question and response can then be used as an instant survey tool to get at those varying attitudes and to make the class as a whole aware that students differ in their perspectives.  The second is to pose a question that has a right answer and see how the class does in response.  If many of the responses are wrong but some are right, this provides a launch point for small groups of students to discuss the question among themselves.  Which is the right answer and why?  This process "works" if a redo of the clicker question then gets many more students to select the right answer.

Why do students participate in the clicker exercise rather than opt out?  For the most part this is because they get some course points for answering the question.  In the polling sort of questions, this must be participation credit, since there is no right answer.  Otherwise, credit could potentially depend on what answer the student supplied.  In the human sexuality course, there were some polling questions of a very personal nature posed.  Those were done anonymously - no credit.  Students then participate on a volunteer basis.  All students are interested in seeing the class distribution across these personal questions.  They volunteer their answers both because they feel safe in doing so and because it is the good citizen thing to do, since they understand the benefit from seeing the class distribution.

Several of the panelists report that the students want their clicker points.  It is a strong motivator.  This was true even in the human sexuality class, though here I believe we're referring to motivation to answer the clicker questions as distinct from motivation to follow (or attend) the lecture.  It also seemed, across the various presentations, that this student motivation for clicker points was there prior to them taking that particular class.  Indeed, Julie Shapland said that the students really want these points although she allocates so few of the course total in this manner that it may be irrational on the part of the students to participate with the clickers - there is a high likelihood that such participation will have no impact on their final course grade.

Now I want to return to my earlier question:  Is there inherent interest in technical subject matter?  Let me try to answer this via example, relying on my experience teaching intermediate microeconomics, a required course for students in the College of Business.  Most of them dislike, if not totally detest the course.  Yet the subject matter in intermediate microeconomics forms the basis of many of the things they will study.  Economic cost notions are extraordinarily important in accounting, consumer theory is the basis of much of the study of marketing, and to understand risk premiums in finance, one really needs to be steeped in the elements of decision making under uncertainty.  So there are reasons why Business students should find intermediate micro compelling, and a few do, but most do not.  The main reason is the style in which it is taught.  It entails a lot of math modeling, which these students are not getting in their other courses.  The students have all had the math prerequisite courses.  But most haven't internalized the subject matter of those courses in a way where it is second nature to use those tools for the economics.  Consequently, the entire course becomes a struggle and students don't want to struggle when they can't see the relevance in doing so.

How much of the rest of the courses that non-Business students take looks like intermediate microeconomics to Business students?  I don't know.  In my own recent teaching, an upper level undergraduate economics course where most of the students are Econ majors, I may be getting a biased view of the situation, because many Econ majors are Business major wannabes.  Biased or not, this is my general impression.

Students may claim inherent interest in a subject, particularly in their major, but my discipline suggests, via a theory called revealed preference, that it is far better to observe the student behavior and infer the preference from that then to solicit preference information directly.  I don't see freshmen and sophomores in my class.  The juniors and seniors who do take the course are for the most part so caught up in the quest for a good grade that it trumps any other possible motivation.  Since I do have the students write on a weekly basis, (some of) their thinking about course subject matter is more transparent to me as an instructor than it would be to most instructors in other upper level courses that don't have students do this writing.  For the most part their efforts reflect a mild interest in the subject, at best, and if they are expecting to get an "A" grade they are content with their efforts, and don't reflect at all on whether they might understand the subject at a deeper level if they spent more time on it.  Such deeper understanding doesn't appear to be a value in itself for the students.  But if students had inherent interest for the subject, my belief is that the students would crave a deeper understanding.

So my principal concern with the use of clickers is that it leverages the student desire for course points in a way where there is no countervailing activity that might appeal to inherent interest in the subject.  Multiple times during the session, the panelists stressed the clickers must be used in a way that shows what the instructor cares about.  This was in reference to the type of questions that are asked and how those questions tie into the lecture.  This makes sense and is an important point.  Yet the fact remains, the "currency" that makes use of the clickers work is the reward in course points.  Large class instructors care about their grading scheme in that it be fair, administered well, and motivate the students.  But they don't care about the points as currency the way they care about the subject matter they teach.  And that they rely on points as currency to communicate may severely limit their ability to communicate what they do care about the subject matter.

In my class, one thing I do is to post on occasion snippets of news articles along with links so students can find the full story.  Most of these posts are on topics that are related to course themes.  (When the Nobel in Economics is awarded, I also post about that.)  I give such posts a tag - Econ in the News.  It is very rare that they get comments and the access stats show quite limited access.  Ditto for my early PowerPoint presentations and associated videos.  Neither of these have assessments associated with them.  They are there to give the interested student access to an expanded set of materials relevant to the course.  In contrast, posts about the homework do get comments and are accessed a lot.  Yet by my earlier argument about revealed preference, the fact that I do create a good bit of non-assessed content and post that to the course site is indicative that I care about that stuff.  Students who access the course site on a regular basis can't help but notice that.

Several year ago I posted a critique of a video on motivation that featured the voice of Daniel Pink, both the extrinsic and intrinsic variety of motivation.  Pink's main point is that knowledge work is unlike manual labor in regard to motivation.  For the former, intrinsic motivation is a much more powerful motivator than any possible sort of financial reward.  Further, explicit pay for performance schemes are apt to backfire for knowledge work, where they do work well for manual labor, because such schemes serve as a distraction from the inherent interest in the problem at hand.  My critique of Pink was not based on this observation.  I concur on this point.  Rather it was based on his overly simple view of the economics.  He argued that pay should be used to initiate activity and then it should be generous, so it stays out of mind thereafter.

Even without explicit pay for performance, however, there are still salary increases and promotions to consider and the manager may also have to confront that the employee's external job market has improved dramatically as a consequence of success on the job in an earlier project.  In other words, re-initiation happens periodically, often for exogenous reasons, and may very well happen while the employee is immersed in a highly interesting project.  That's a real part of work life.  But taking Pink's point for where it is sensible, it is folly to have more frequent adjustments of pay based on performance, done by the whim of the manager rather than as a result of external factors.

Yet the analog is what we seem to do in teaching.  And with the clickers, where there are perhaps five questions in a fifty minute session, the frequency of the extrinsic reward is quite high.  Is it then interfering with intrinsic motivation for the subject that students might have?

It is unfair of me to lay this issue at the feet of the large class instructors.  They face many logistics hurdles on the way to teaching a tolerably good course. It rightfully is an issue that the entire campus should take on.  And thought that way, the intrinsic motivation issue has been addressed largely, to date, via outside-of-class means.  Students can engage in undergraduate research.  Or they can get involved in a registered student organization with an academic/intellectual orientation.  The undergraduate research option might be quite enriching for those students who do it, but how many do, particularly in the social sciences?  I suspect comparatively few.  And I suspect something quite similar for RSO activity that both inherently interests the student and promotes the students' intellectual growth.

In contrast, courses are universal and we think of them as the main place where students learn while in college.  Motivation is a key element in learning, for anyone, for me as an adult learner just as much as as for our students.  If inherently interested in a subject, an adult learner can then often teach himself, get expert help on as needed basis, but otherwise be self-directed.  One important goal of undergraduate education is to prepare students to be effective adult learners.  To the extent that we are relying exclusively on extrinsic rewards to motivate our undergraduates, course points and class grades, we will consistently fall short of this goal. 

Clickers are comparatively new technology.  But they are steeped in reward schemes that really are outdated.  It is true that the students who sit in the back of the lecture wouldn't engage without the clickers and some of them wouldn't come at all.  For that reason, I'm not saying to do away with the clickers.  I'm saying they are not sufficient.  Think of the students who sit in the front and would come to class even without the clickers.  What might we do to raise their inherent interest in the subject?  Some instructors may even have an answer to that question.  Are other instructors aware of those answers?  I, for one, am not.

In the previous decade I taught infrequently and when I did it was mostly for the Campus Honors Program.  Those students behave in a way that I'd like to see emulated by the students who sit in the front of the large lectures.  What can we do to achieve that as a goal?  That is the question we should be asking. 

Where is humorous satire when we need it?

The Cuban Missile Crisis was in 1962.  Dr. Strangelove came out in 1964.  The Internet and the 24-hour news cycle has sped everything up.

In his column today Roger Cohen weaves a fictitious, but plausible, and definitely not funny, story about how World War III might start, updating the Assassination of the Archduke Ferdinand to match contemporary events in the Ukraine.  After reading the piece, I desperately wanted a Kubrick-like alternative to put me at ease.  Doesn't it seem there should be something in that vein in tomorrow's paper?  If you can make fun of it then it's not so scary.

On a different note, you might think it's in the NY Times long-run interest to try to lure current college students and recent grads who get their news from The Daily Show and Colbert to read their newspaper on occasion.  What is in the paper now to attract this audience?   Most of the TV humor on these show is burlesque-like.  On that level the Times can't compete.  Instead, might the written form offer up wit with pith?  At present Gail Collins is the only columnist to venture into this terrain.  I love her writing for that.  But her style reminds one of Bob and Ray.  Maybe these potential readers require a bit more edge.  Where are the latter day Mort Sahls and Dick Gregorys?  Would some of the up and comers in this vogue be willing to deliver their material in written form?  And would the Times be willing to publish it if they were?

Then there is the geriatric set, which I am to soon enter.  (Sixty is the new seventy.)  They have their own tsoris.  They don't need the news to pile it on.  But Soaps, Oprah, and Dr. Phil are not the answer.  The Onion appeals to a younger crowd.  Might not Larry Doyle or someone else of his ilk be the ticket? There is a need to fill here.

To borrow from Gail Collins, "Come on people!"

Instinct and Tâtonnement

For my non-economist readers who are likely unfamiliar with the term Tâtonnement, I learned it in the second quarter of graduate school while studying microeconomics.  Léon Walras, the father of General Equilibrium theory, used it to explain the process by which his fictitious auctioneer would arrive at equilibrium prices.  I was taught that it meant "groping."  Google Translate says it means "grope."  I prefer the gerund to the noun in this case as the latter refers to a single instance while the former conveys the sense of an ongoing process.

I am using it in my title to give a nod to my economics training while referring to what I go through in writing my blog posts.  To make this explicit for the reader here, though I don't actually do this in a conscious way while writing, there are three fundamental questions I'm asking, each borrowed from the modeling in economics:  What are the axioms?  What is the model?  What theorems result?  (What conclusions can be drawn?)  These are posed simultaneously, not sequentially.  The art is in finding something that works or at least seems to upon initial inspection.  The writing itself then becomes a more detailed inspection and elaboration of the argument.  It is what I was trained to do and what I continue to do all these years later, in spite of having stopped doing formal economic modeling long ago.  There is, of course, a second reason for using the French term rather than the English gerund.  Try substituting the the latter for the former in my title and see what mental picture that creates.  It is not what I have in mind, at least not in the crass way that such a title would suggest.

On Friday David Brooks had a column, The Deepest Self, that challenged me.  I no longer read Brooks' column on a regular basis.  I don't like the way he argues - too preachy - and some of his maintained assumptions seem wrong to me.  But sometimes he hits on themes that I think are important.  This column was one of those times.  It is about the tension between our primitive nature, with its evolutionary basis, and our capacity to rise above the mire via our creativity.  This paragraph was my favorite from the piece.  It is where I agree with Brooks.

In fact, while we are animals, we have much higher opportunities. While we start with and are influenced by evolutionary forces, people also have the chance to make themselves deep in a way not explicable in strictly evolutionary terms. 

The rest of his treatment of the subject I found less satisfactory.  Rather than take him on, point by point, I will simply use this as a launch point for my own inquiry.  I want to do this at the individual level, where I will take a personal perspective to address the issues.  Perhaps this should be done at the social level too.  I may do that in a subsequent post.

On Friday morning I was getting geared up to watch (on TV) the Illini play Michigan in the quarterfinals of the Big Ten basketball tournament.  While in the past I've been a rabid fan, making it to Rupp Arena to see the Illini play Kentucky in the Elite Eight of the NCAA Tournament in 1984, going to all three sites the Illini played in during their run to the Final Four in 1989, more recently some of that has cooled.  Part of that is simply that sitting in the seats at the Assembly Hall (I know it is now called the Scott Trade Center, but until the renovation is complete I will call it the Assembly Hall) is physically uncomfortable for me now.  My joints much prefer sitting on the couch in front of the TV.  The other part is that I became a fly-by-night fan the past few years in response to the team's mid season swoons.  As the team headed into its tailspin during the Big Ten season, I would start watching games but then turn them off after I got disgusted with the performance.  After several losses in a row I stopped watching altogether, though I would keep track of the score on the ESPN scoreboard.

However, as the team seemed to be on an uptick I started to watch again.  The game at Iowa, the season finale, was great viewing.  The game with Indiana on Thursday in the first round of the Big Ten tournament was not quite as climactic as the Iowas game, but it was quite intense through much of it and very enjoyable to watch.  I was therefore looking forward to the game with Michigan, in spite of the thrashing they had given us ten days earlier.  There is the added bit that my brother works at Michigan and he had been a bit obnoxious about the first game in email.  The sibling rivalry is not close to what it was when we shared a room growing up, but it isn't entirely absent either.

Being a sports fan has a libidinous aspect.  It conjures up an image of Romans at the Coliseum, watching the gladiators fight to the death.  I first heard it put this way by Mort Kamien, who had been my professor in the first quarter of microeconomics in graduate school (partial equilibrium, not general equilibrium) and then again in the third quarter in a course on dynamic optimization co-taught with Nancy Schwartz.  But I didn't hear it from him until ten or fifteen years later when he came to Champaign to give a talk.  He mentioned it beforehand while we were schmoozing and reminiscing.  And he put it in quite a disparaging way.  Real academics may tolerate big time sports at their universities, understanding that it part of the way these institutions remain viable, but they don't become fans themselves.  That would detract from their research, which should command their total focus.  It is that attitude, implicit in the Brooks piece, which I want to take on here.

The game with Michigan was as good as it gets.  During the first half Michigan seemed the better team, but Illinois kept it close.  At the start of the second half Michigan went on a run and threatened to break the game open.  Our coach decide to on defense play a two-three zone and it eventually put Michigan into a funk.  On offense, he told the team to take the ball to the rim and avoid the outside shot.  We started to click that way, got their starting center into foul trouble, and the momentum had clearly swung our way.  We closed the gap and eventually took a small lead.  They were more skilled.  We were tougher.  It seemed as if toughness would have its day.  I'm sure everybody in the arena felt that same thing. Michigan did make a bit of a comeback and with under four seconds left they had a one-point lead.  But we were able to drive the ball to the rim with our toughest player, Tracy Abrams, getting the last shot off.  It was his favorite shot, a little floater.  It missed.  That was the ballgame.

I had intended to write this piece on Friday, after the game concluded.  But I was down in the dumps then and needed a whole day to recover, then a good part of another day to reconcile that immediate experience with what I want to say on the subject raised in Brooks' column.  Earlier this morning I did a Google search on sports fans and the libido.  In the middle of the page I found a link to this page, which seems to confirm the hypothesis on the biological front.  Being an intense fan, even one who watches at home like I do, but who claps and yells encouragement to the team as if at the arena, has an arousing effect.  It tends to raise testosterone levels. (Presumably the studies referred to focused on male fans.)  The piece then argues that can be either good or bad, making the point that a little bit of that might be beneficial, but there can be too much of a good thing.  In other words, being a sports fan is something of a roller coaster.  Sometimes the ride is great.  Other times it can be overwhelming.

My experience writing, particularly writing this blog, has changed over the years.  When I started, in 2005, I had such a backlog of issues that were weighing on my mind that most of the posts simply flowed out.   After a year or so, there was the occasional struggle, such as this post on The Virtual University Dilemma, but that was because at the time I held a position of responsibility for the Campus and I was writing on a then politically sensitive topic.  My University was then contemplating a venture with a online Global Campus, something that eventually failed.  Later as I moved to the College of Business and then retired, this particular reason to pull my punches lessened substantially.  But I strive to not be overly repetitive, to produce some value add for my readers, and for myself to feel I've gotten something of substance out of the effort.  Sometimes meeting those goals is quite a challenge.  Then I struggle for that reason.  And when I struggle the emotional aspect that accompanies the writing is very similar to what it's like being a sports fan.  Being stuck on a writing issue is very much like watching the team when it falls behind.  There is disappointment, maybe anger.  When after quite a while, a possible way out emerges, there is an emotional boost.  If it pans out, whether that comes in a quick AHA or a more deliberate checking of the possibilities, there is elation.  Though the activity is intellectual at its core, the libido is present, big time.

So the message I want to give here is that trying to repress the libido so as to spend more time thinking deep thoughts is folly.  It is part and parcel of the activity.  It will be present any time the participant cares deeply about the outcome, regardless of what the activity is.  There is a related question, of the role recreational diversion plays in creative activity, such as writing, if one sees the libido expressed in both.

To this I would respond that the goal is some semblance of balance.  It is impossible to stop the roller coaster entirely, but one doesn't want to go off the deep end with any frequency, if at all. The derivative benefit, then, from the recreational diversion in that it gives more experience with expression of the libido and therefore a better understanding of oneself over time.  With that there is the knowledge gained about when self-discipline is called for, as Brooks argues and when self-comfort is more appropriate, something Brooks implicitly frowns on.   There is the further benefit that if one attempts writing on a regular basis then one needs fallow periods as well, sometimes during the writing activity as a way for ideas to percolate, other times in between efforts so as to come to the next project fresh and eager.  Recreational diversion serve those functions well.

Early high touch intellectual experiences for students

One of the peculiarities of any student information system, as an instructor sees the data in it, is the profile information available about each student.  Major and year in school (freshman, sophomore, etc.) might seem like obvious things to include and in my experience they are.  I wish they also included prior courses taken and grades in those courses, as it would give a better picture of the student's preparation as well as provide an indicator of the student's grade expectation.  But that information is FERPA protected, so instructors can't see it.  Academic Advisors can see it.  This sort of difference in access to profile information can't possibly be in the law itself.  Instead, I believe it results from institutional policy, the mindset of which seems to say that instructors don't advise and they are to teach students irrespective of their prior preparation.  It is the student's call whether they are prepared for the class or not.

I am making a thing of this, though I am resigned to the fact that the situation will not change, just to emphasize what I say next.  The student's home address is available to instructors in our SIS.  I have no idea why.  In principle, this allows the instructor to know which students are from the Chicago suburbs, which are from the city itself, which are from downstate, and which are from out of state.  In general, I ignore this information, but for a student who has come to my attention, for good or bad, I will sometimes look at it to see if it correlates with what I have observed.  For example, suppose I have an out-of-class meeting scheduled with a student who then blows it off.  If I find the kid is from a well-to-do suburb that fact helps to complete the picture to explain what's going on.  At an entirely different level, the home address information provides a reasonably good indicator of what the student's family is paying in tuition. 

That different students pay different tuition rates for what seems to be the same service amounts to price discrimination, though one must be careful in thinking about this.  Are there really cost differentials in serving the different students that explains the difference in tuition rates?  For example, the difference in tuition rates between in state and out of state students can be partly explained by the former receiving a subsidy from tax revenues while the latter do not.  The price discrimination piece is only that part of the out of state tuition which can't be explained by the lack of subsidy.  My sense is that such price discrimination has increased over time as part and parcel of the rise in tuition more broadly and the influx of international students at the undergraduate level.  It is one way the university has remained financially afloat as the tax revenue dollars have not kept pace with the rest of university expenditure. 

Price discrimination often comes with quality variation as part of the deal.  One might feel it anathema that quality variation in the education supplied be based on the home address of the student.  Yet there is other type of price discrimination with associated quality variation at the university where most people hardly raise an eyebrow.  One example is differential tuition rates across colleges, with some colleges requiring a tuition surcharge over the base tuition while Liberal Arts and Sciences, in particular, does not.  The colleges with a surcharge typically have better facilities, better ancillary services, a higher expected wage upon graduation, and perhaps other quality enhancements.  They also have higher average standardized test scores of the entering students, which suggests these quality "rewards" are merit based rather than tuition based.  I will return to the issue of merit rewards below.

Perhaps surprisingly to most people, until they think about a bit, price discrimination happens even under apparently uniform pricing, because of quality variation in delivery.  Upperclassman typically are in smaller classes and are more likely taught by tenure track faculty.  Freshmen are more apt to be in large classes, now frequently taught by adjunct instructors, and have graduate students as TAs.  Indeed, when I first started at Illinois there was a tuition differential by class rank.  Seniors paid more than freshman.  At some point they got rid of that differential, the consequence being that we now have flat tuition for all four ranks (though it does vary by the year a student starts) but with quality generally rising as the student gets further along in her studies.  For the same reasons, there is a kind of price discrimination in favor of students from well off suburban high schools and against students from poor rural and inner city schools.  The suburban kids typically get to take a lot of AP classes and if they score well enough on the AP exam then get to bypass that particular large lecture class.  They also are apt to graduate earlier and therefore pay less tuition overall.

Let me turn to non-tuition based variation in educational quality that the institution embraces.  One of these is via the various Honors programs - there are programs at both the campus level and the college level - which are targeted at the very high performers among the students.  So Honors programs fall into the merit reward category.  In the previous decade I taught three times in the Campus Honors Program.  Small seminar classes with very bright and engaged students are a joy to teach, but there is a reason why most of the rest of our course offerings are not taught in a small seminar.  Then, even with the very limited number of CHP offerings, the model was for faculty to teach a CHP class is to do so as an overload and earn a modest stipend from that.  Further, because the size of the program is quite small, there is the question of where the line is drawn for admission to the program and if there are quite meritorious students who don't get in because space is limited.  The last time I taught a CHP class, in fall 2009, I queried the class on this point.  The consensus was that in their regular classes there were some students who did just as well as they did but were not in Honors.  For illustration, though I'm making up these numbers, if CHP includes the top 2% of the class, then perhaps the top 10% of the class is meritorious. Additionally,  there is some measurement error in properly coming up with rankings like these, so who gets in can be a bit arbitrary.

Then there are non-tuition based interventions targeted at disadvantaged students.  I participated in one of these for I-Promise students, as a mentor.   All mentors volunteer, but as far as I know volunteer mentoring is not generally available to students.  In this particular case the idea of the mentoring is to focus on the first year the student is on campus.  If the student can get over the hump of that first year, then the student is apt to get through.  The I-Promise kids are academically talented, but from low income families.  The mentoring is aimed at helping the students negotiate through some issues that most other students won't confront at all or, if they do confront them, then they have other buffers to manage the issues.  That rationalizes the program as is.   But one might reasonably ask whether students from reasonably well off families might benefit from mentoring too, thinking that there are more dimensions to student performance than not dropping out of school.  If so, why don't we try to do something in this arena?

There is another way instructional quality varies from student to student and, indeed, such variation happens within an individual class, at least in my class.  Of course, some of the instruction is pushing stuff at the students and in that sense the effort is uniform.  But another part of the instruction is reciprocation for something that the student initiated.  Students who initiate a lot get more and better reciprocation.   Passive students who initiate hardly or not at all don't get as much of my time.  Such initiation can happen in many different forms.  One is by asking questions during class.  Another is by asking questions online outside of class.  A third is by attending office hours on a regular basis.  A fourth is by making an unusually good contribution in completing course work.  Still another comes from completing assignments early and getting noticed for that.  Maybe sitting in the front row is a form of initiation.  (Surely sitting in the back is the opposite, though my classroom last semester was pretty small.)

If all student behavior were rational, if it were understood that initiation by the student would bring out some reciprocation by the instructor, if that reciprocation were perceived by the students as higher quality of instruction, and if the students wanted higher quality of instruction as long as it didn't impact their pocketbook, then one should expect the total capacity of the instructor to reciprocate to get used up and that the students would compete for that bit of reciprocation which is directed at themselves.

This has not been my recent teaching experience.  Based on what I have observed in the past three years at least one and possibly all of those conditionals don't hold true, as a general matter.  In contrast, a handful of students do act in accord with the model described in the previous paragraph.  They have been socialized into acting responsibly about their own education.  The system works for them.  It doesn't work so well for the others, the majority of students.  What should be done about it?

High touch, face to face interaction with a credible peer, is the answer.   You don't have to take my word on this.  Atul Gawande has a really excellent essay on the diffusion of innovation from last summer called Slow Ideas, which makes this essential point.  His context is different from what we're considering here.  Gawande is concerned with proper infant care by mothers in developing countries.  The mother is unaware of healthful approaches and often does things that are harmful to the baby, but the mother does this out of ignorance, not out of anger. If the mother understood a better approach, she'd embrace it.  One would think the analogous thing should be true for our undergraduates.

In anticipation of Gawande, really in borrowing results from 1990s about technology innovation in high enrollment courses, where to me the most interesting single capability the technology enabled was to put undergrads who had taken the course before in a role of responsibility in assisting students currently taking the course, I wrote a series of blog posts back in 2005 entitled Inward Looking Service Learning (there are 7 posts and if all of them read they should be read chronologically, though they are displayed in reverse chronological order at the link) which sketched how a solution scalable to our campus might be attained.  I still subscribe to the thesis, though the particular mechanism in those posts, the focus was on the study group and peer-mentors would interact with current students in that setting, might not be as effective as one-on-one interaction.  However, one-on-one interaction would be much harder to manage at scale.  In other words, there needs to be a lot of experimenting on the right way to do this.

Nearly nine years after those INSL posts, however, nobody seems to be embracing the idea.  There are a handful of dedicated instructors who subscribe to peer mentoring in their own teaching, finding the approach via their own research, not though my blog.  But the peer mentoring idea is not diffusing at all beyond that, slowly or otherwise.  And in its current usage peer mentoring is about teaching a particular class better.  It isn't (yet) about socializing students to become more active about their own learning in all the classes they take.   Higher ups and people in the learning technology field alike don't seem to be thinking of the issue this way. 

The suggestions below are meant as ways to raise awareness of what early high touch approaches might accomplish.  One of the issues here is that doing anything novel that otherwise doesn't reduce obligations from elsewhere can only be done on a volunteer basis, or possibly on an overload basis if some non-recurrent dollars can be found to support experiments in this area.

1. Find a group of willing instructors who teach high enrollment courses that are taken primarily by first-year students and do such teaching on a regular basis.  The disciplines of these courses can vary.  Have these instructors form a discussion group based on the following.  Each instructor will have a high touch discussion section that they will run - say with 5 students or 10 students at most.  Have the students opt into the high touch section, with the understanding ahead of time that they will have some intensive interaction with the course instructor.  Let the experiences in teaching these sections be the basis of the discussions the instructors have with one another.  Because of the selection bias from how students opt into these sections, avoid trying to compare performance of the students in the high touch section with the rest of the students in the course.  The focus should be on whether the students in the high touch section seem to grow over the semester and also on whether the instructors get some insight into their own teaching as a consequence of these interactions.  
2. Do something similar with Teaching Assistants.  In addition to their other sections, let them have one that is high touch (here I'd suggest no more than 5 students in that section).  Have each TA in some course do this and then have the TAs interact about this experiences in the high touch sections in a similar manner to how the instructors interact in the discussion group. 
3. In both of the above this would be new and the fear is that the participants would revert to the old way of doing things - they'd lecture to the few students who are there or if not that then totally dominate the conversation when in one-on-one discussion with students.  This can be anticipated.  I am not sure what prior preparation would best discourage it, but perhaps some modeling of peer mentoring by current practitioners should be made available to the participants ahead of time so they have some mental model ahead of time of what effective practice looks like.  
4. There needs to be several neutral observers who can both write about what happens in these experiments and communicate to the powers that be about whether it is worthwhile to do (fund) additional rounds of experimentation.  These observers should also engage in focus groups with the students.  The students, perhaps more than the instructors, might be better able to suggest changes in practice that would encourage them to improve their learning strategies, this in spite of the selection bias mentioned above, because the students will have perspective about the consequences of taking other courses where the high touch approach is not embraced.  Anyone who confronts the scaling of such a high touch approach will have to reckon with the fact that students who participate early on will mainly be operating in the traditional environment and that might overwhelm high touch interventions which would be effective if they were more prevalent.

Let me close with some acknowledgements.  I had lunch with Joe Hinchliffe on Tuesday and with Burks Oakley on Wednesday and both of them made comments during the conversation that pointed to an embrace of a high touch approach.  So I thank them for that and hope to get a few more folks on the bandwagon soon.  If anyone out there would like to chat about this further - face to face or virtually, I'd be delighted to do so. 

Assessing student writing - my take

In today's NY Times the Room for Debate series features essays on the topic, Can Writing Be Assessed?  I normally eschew Room for Debate.  Each essay is such a quick hitter that there is barely time to make a point before it is done.  There is not much depth generated this way, especially since the various authors are not responding to one another.  They simply offer up their own take on the subject.

Nevertheless, the subject matter interested me so I read the essays at the link above.  There were some reasonable points made, but I thought the treatment inadequate in sum.  So I will cover here the issues as I see them, both for assessment of writing on a standardized exam, like the SAT, and assessment of writing during regular instruction, as well as the relationship between the two.

Let me begin with what we mean when we use the word "assess."  The following is taken directly from Dictionary.com.

as·sess
[uh-ses] Show IPA
verb (used with object)
1.to estimate officially the value of (property, income, etc.) as a basis for taxation.
2.to fix or determine the amount of (damages, a tax, a fine, etc.): The hurricane damage was assessed at six million dollars.
3.to impose a tax or other charge on.
4.to estimate or judge the value, character, etc., of; evaluate: to assess one's efforts.

Definition #4 seems most apropos.  I like the use of the word estimate in the definition.  It suggests some lack of precision, such as that different people assessing the same work might come to somewhat different conclusions.   It also suggests that some variation across works may nonetheless produce the same assessment.  I will concentrate on this second issue here and ask whether variation in the works is a good or bad thing.

Let me start with a very mundane example to get the ball rolling and take the issue outside of the school setting so as not to make it too difficult to consider.  When somebody has a life event, Facebook friends will often post something to that person's Wall.  My experience with reading these is that there is very little meat to these postings and most say essentially the same thing.   I'm going to assume most of my readers will have observed something quite similar.  I want to note that as a writer of such Wall posts, I treat a birthday quite differently than I treat a death of a loved one, though most other people don't make this sort of distinction.  My general view is that the writer's job is to make a novel contribution rather than echo what everyone else has said.  If I know the person well enough that I'm confident they won't think ill of me for trying, I will deliberately attempt to differentiate my message from the rest of the crowd, with a pun, a stab at something clever, or just being silly.  On more solemn occasions, this sort of playfulness doesn't seem appropriate, and here the message is simply showing support, so is much more alike the other posts.

This, then, is the first issue with assessing writing.  Is variation across writing pieces produced by different students desired or not?  I have never queried my Facebook friends on this so I don't know the answer, but let me suggest there are two possible ways to consider the birthday messages.  One is that most other people think that too is a time to show support and that is what they are doing with their posts.  I disagree with that, but surely it is one possible explanation for what is observed.  This brings to mind a recent piece in Slate, Inside the Box: People don't actually like creativity.  A different possible explanation is that most people would prefer to generate a witty, original post, especially if they could do so in the time it takes to write the post that they end up with, but they don't practice doing such writing so view it out of reach.

It would be absurd, of course, to assign letter grades to posts in Facebook on somebody else's wall  (or on one's own wall, for that matter).  But let's note that if we took this counter factual as the norm, then two different grading schemes would emerge and which is the right one depends on whether variation is perceived as a good or bad thing.  If it is bad then on those birthday posts most everyone else gets an A, while I get a B or C.  That I do generate such variation signals I believe the other grading scheme to be the better one, in this case.

Now let me elevate from the mundane to newspaper writing and note that as a reader with straight news you'd expect little variation from one reporter to the next writing the piece, with what variation is observed explained perhaps by variation in the legwork done to generate the story or by modest stylistic differences from one reporter to the next that the editor will tolerate as it adds flavor to the piece.  In contrast, you'd expect much more variation from reading Op-Ed pieces, such as the essays in this Room for Debate discussion on assessing student writing.  Each essay reflects the author's worldview, which in turn depends on the author's prior thinking and relevant experience.  This makes the writing much more idiosyncratic, a good thing if you like to read Op-Ed, as I do.

Presumably we'd like students to be able to write in both ways, as a neutral observer and as one who has a considered opinion on the matter.  I have not reviewed sample writing questions from the SAT in preparing to write this piece, so I could be wrong here, but I suspect there is bias in the testing in favor of the neutral observer type of writing.  This is particularly problematic for the teaching of writing, since the book report is an early form that students produce in elementary school, and in the conclusion the student is to say whether she liked the book or not, presumably with a sentence or two of justification as to why.  To be a neutral observer and yet care about what is being observed takes sophistication and a lot of practice.  What I suspect happens all too often is that the student is tasked with being a neutral observer but has only extrinsic motivation, provided by the assessment itself.  That is not a good way to teach.

Let me turn to the next issue, suggested in the previous paragraph, which relates the assessment of writing to the assessment of reading.  The SAT and other standardized tests treat these as separate assessments.  This means it is possible for a student to do well on one and poorly on the other.  Abstracting from English as a second language issues, and possible other explanations that can rationalize such a negative correlation, how is one to make sense of somebody who seems to be a good writer but a poor reader or vice versa?  To me, there should be a strong positive correlation between the two.  And even if on occasion one finds a student writes well but reads poorly, shouldn't reading and writing be assessed holistically rather than separately?

This amounts to asking the following.  Is it intellectually the same thing to give an interpretation and a conclusion of what a reading passage says, writing up both the interpretation and conclusion, as it is selecting one answer in a multiple choice question, where each answer proffered provides an interpretation and conclusion?  I don't know but this is what I surmise.  Writing up the answer there is far less doubt about what the student is thinking - as long as the student is comfortable expressing herself in writing.  With the multiple choice question, the student's thinking is masked.  For right answers we can't tell whether the student knew it or got lucky while for wrong answers we can't tell whether the student thought it was the right answer or merely guessed.   (I should add here that asking the student to explain aloud the interpretation and conclusion from the reading passage is similar if not identical to getting such an explanation in writing and may be preferred to writing for on the spot diagnosis of any fundamental misunderstandings.)  I am quite fearful that there are many good test takers who nevertheless can't provide good written explanations of what they've read.

This gets me to the final point.  One larger goal for K-12 education is that students learn to read and write for themselves, meaning the students find these activities engaging so participate in them willingly.  This means the student will do these activities in and outside the school setting.  It also means the student will engage in these activities to promote her own personal growth.  The standardized testing might be neutral on this front if it weren't so heavily emphasized.  But with it's emphasis, we likely are getting perverse negative reactions - these activities become for school only, so students plateau far too early in their skills as both readers and as writers.

One wonders whether school might provide a different sort of assessment that is not test prep, with weekly reading and writing projects, for which the primary goal is that students do them.  Feedback would be important to encourage student involvement and suggest ways for improvement.  But letter (or numerical) grades could quite possibly be counterproductive, producing the same sort of reaction that the standardized tests produce.  The focus would be on student motivation for the doing.  There is an approach now to keep students in lock step in their reading.  That would have to be discarded in favor of an individualized approach. The latter would be much better to sustain student interest.

It seems to me we could do much better this way.  But then, what I find obvious doesn't seem to have as much appeal for the crowd. 

Getting Testy About the SAT - Wrongs of Passage

Though the author of the piece linked below was an English major, while I was a math guy in high school, I pretty much agree with her assessment of the SAT.  Let me add a bit from my perch.

On the test itself, the math side is less about memorization than the verbal portion of the exam.  There is reasoning through algebra and geometry problems, and the beloved word problems, each of which need to be translated into an algebra or geometry problem to obtain a solution.  I wish that had been the entire test when I took it, but for some reason the authors felt it necessary to have one or two questions about Roman numerals.  I know them through 100 and understand the schema for representation, but confuse M and D, always have.  It seemed to me silly to have a question about that then and I haven't changed my mind about it since.  Then there were also questions of the type: 1, 4, 9, 16, what is the next number in the sequence?  These are pattern recognition questions.  There is an intellectual issue with these sort of questions - more than one pattern can be found to explain the sequence so far.  In the above, for example, the pattern might be odd, even, odd, even, with the numbers increasing, rather than each number having to be a perfect square (the latter being what the SAT would say is the "right" answer).  So the math test is not without criticism, but it's weakness are probably less severe than the weakness of the verbal exam.

On the consequences of the exam for the self-esteem of the student, the author appears entirely okay on grades in school, both as measure and motivator.  I am less okay on that as I believe it does quite a bit of damage to kids who don't rise above all that grading and figure out to learn for learning's sake.  (See my post College without Grades.)  Given that the grading system itself creates a lot of stress on the students, much needless in my view, what does the SAT contribute to make the problem worse?  Here, I believe the real issue happens when the grades as signal give a different message than what the SAT scores as signal provide.  In particular, students with a high GPA at a reasonably good school but who perform in a mediocre way on the SAT take a real blow to the ego.  No good comes of that.  That exam score, known to teachers and classmates alike, becomes that student's personal Mark of Cain.

There is then the issue of whether the SAT predicts college performance.  I will cite a few bits of different sorts of evidence.  First, for a couple of years I was on the Campus Honors Program Advisory Committee and one part of that was to review the applications of students to the program to determine who will get in.  Many of the applications I reviewed (not a great number in total with each an interest in economics or business) were from students who had perfect standardized test scores, whether SAT or ACT or both.  Students also had a variety of extra curricular activities which, to my non-discerning eyes, distinguished them from one another not at all.  What was left as a differentiator were the rather short essays.  (The U of I doesn't seem to require letters of recommendation or those weren't included in the packets we reviewed.)  I should add here that students don't apply for CHP - the applications are singled out based on other factors (the high standardized test scores and/or the geography of where the application is from).  So the kids may not have put in much effort into writing the essays.  Alternatively, insufficient guidelines were provided to inform them about what would please me as a reader.  With those caveats, I was totally underwhelmed by the writing, with very few exceptions to the rule.  So on the one hand you had these great test scores while on the other hand you had these mediocre essays.  What should be concluded from that?

Next, before I attended the Frye Leadership Institute in June 2003, we attendees were asked to read The Gatekeepers by Jacques Steinberg, which is an in depth look at the admission practices at Wesleyan University.  One gets the impression from reading that book that places like Wesleyan really do sort through a wide variety of information about each potential student.  Given that, perhaps the standardized test scores at such a place provide little value add to the overall picture of the student.  But what about places like Illinois, where the volume of applications has to be more than an order of magnitude greater than what it is at Wesleyan, and where such in depth looking at students doesn't happen.  (Most students who apply to Illinois do not have an interview with an admissions officer.)  Could Illinois have a reasonable admissions process if there were no standardized testing?

Then I'd like to talk specifically about performance on the math part of the test.  When I was a graduate student my adviser told me that the math part of the GRE (very similar to the math part of the SAT) was a good predictor of performance in Econ graduate school (which is very demanding about the math).  Much more recently, I was told something similar about the math score on the ACT as a predictor of performance in General Chemistry.  But my sense of things is that these predictions are not linear.  Outlier good performance on the standardized test (in my recollection of the scoring scale, here I mean above 700 as a score on the Math SAT) does predict quite well good performance in mathy-type courses.  But there is some threshold below which the standardized test performance doesn't predict well at all.  I say this based on my own teaching.  All the students I teach had to have reasonably good standardized test scores to get admitted to the university, but many of the students I've had over the years don't know analytic geometry and can't reason through fairly elementary deductions, even after the math facts have been explained to them.  Somehow there is a way to "fake it" on the standardized test, get a tolerably good score on the math part, yet remain largely math phobic.  If that is right, the scores of such students don't differentiate them from their counterparts, who also are uncomfortable with the math but score less well on the standardized tests. 

Finally, there is the issue of wholesale cheating on the tests, particularly with international students.  For public universities like Illinois, this is a big deal as international students are apt to pay the full out-of-state tuition and thus provide a surplus to the university beyond the cost of their own education.  Revenue strapped public universities then might want to admit more such students, irrespective of how those students will perform once admitted, but they don't want to see a decline in ratings from doing do, hence maintain stern standardized test cutoffs for admission for international students, thereby providing a strong impetus for the wholesale cheating.  The entire situation is ethically murky, at best.  

As long as there are highly selective universities, because a degree from one is perceived as a lifelong advantage, gaming of the system will be the norm.  The SAT is part of that system.  It is easy enough to criticize as an object in itself.  But maybe we should instead be talking about the entire system and ask what can be done for the system to function better and be more humane.  The author of the piece linked below is right that we shouldn't freak out the students.  The SAT does that.  And the rest of the system does it too.  We should be talking about it a lot more. 

Save Us From the SAT - NYTimes.com via kwout