Getting students to do the online work

I'm about three quarters through the new book Freakonomics. Given all the hype, I suppose I should have guessed that it would read more like a long NY Times Sunday Magazine piece than getting insight into Levitt's thinking. There is very little on whether Levitt comes to a new project with a thesis already formed and his look at the data confirms that hypothesis or if he is a naïf at first as he looks at the data and somehow the key idea emerges as he starts to uncover patterns in the data. The only part of the book where the reader is confronted with data of this sort is in the section on (K-12) teacher cheating. Apart from communicating that finding patterns in the data are near impossible without a prior model of what cheating might look like, there is little that suggests how Levitt came to think of this topic. And I don't mean the circumstances - he was asked by the Chicago Public Schools. I mean what was his operating hypothesis, if any, when he first got involved with the project. (I've seen other economists who were school teachers talk about erasing students’ answers on a test when they thought the student had cheated. So I'm fairly confident that Levitt didn't come up with all of this from a blank slate.)

In spite of this deficiency and my conclusion that the vast majority of the book was written by Steven Dubner, the other author, the book is an interesting read in giving background information on a bunch of areas that would not seem to be in the domain of economic analysis. The book does emphasize the people respond to incentives and that is the major orienting point for an economist doing an analysis. And the book spends much of its time talking about "cheating." Apart from the teacher cheating issue, for example, there is an extensive discussion of whether people pay on the honor system put their buck into the coffee can when they take a bagel in the morning. Since the guy who supplied the bagels collected the coffee cans and kept data on that, there was quite a story to tell. That some cheating occurs goes without saying. That there is so much honesty is the mystery for economists, especially without overt enforcement.

Now I want to turn to those ideas of cheating and honesty as they apply to instruction and learning. The bagel guy in Freakonomics has enough data to hypothesize that people are more honest when they feel good - their job is secure, the weather is nice, the office is a fun place to work. But even with all that good feeling, they get maybe 90% honesty. Suppose your standard in the classroom is more than 90%. And suppose you teach a large class that makes it hard to ensure that all the students feel good about the course. Now what?

I know one instructor, when he told me this I thought "that's odd," who rather than use a course management system to have his students do multiple choice homework, has the students fill in bubble sheets outside of class and then bring those bubble sheets to class. Why did he do this? The answer is so the day the homework was due students would attend the live class session. This is what we economists would call an example of "mechanism design." The students respond to incentives. The instructor designs a mechanism that makes the students come to class. This particular mechanism I certainly would not have invented on my own.

I do think that the presentations we've been talking about in the last couple of posts should be blended with online assessment (automated grading) and this is particularly true in large classes. Part of the value of the assessment for the students is so they can benchmark their own learning and see if "they are getting it." But the other part of this type of assessment is simply to monitor the students and assure they are held accountable for their effort. Now the instructor design issue is harder because instead of simply designing presentation, the instructor must design the presentation melded with assessment.

Furthermore, if the instructor teaches the same course repeatedly, either the instructor must change the assessments from one offering or the course to the next. Or the instructor must design the assessment to be cheat proof, in the sense that they are immune from a bright but subversive student posting the answers to a Web site for other students to copy and thereby get the "right answers" without having thought about the underlying questions at all.

My view on these issues are first, that making the combination of presentation and assessment in the form of a dialog: first presentation, then assessment, then more presentation on what what just covered and answered, etc. is a great way to go to make the content engaging. And then on the assessment pieces one needs mutliple versions of similar questions that are hard to catalog in a simple way. A lot of us favor questions that can be solved numerically and then posed with a random number generator. This makes it easier to generate the multiple questions. But otherwise, one needs to write out multiple versions of such questions (and resort to tricks like have the answers appear in random order.)

Generating this type of content, especially in making it high quality, is extremely time consuming. No instructor in their right mind is going to go this route for their entire course. Suddenly teaching a single course has become a lifelong effort. But what about doing enough of this sort of thing so that the instructor has a model of the type of content that is desired. Then the making of this content becomes the alternative to the term paper and it is the students who become the creators. Can that work? I think it has a better chance and should be what we promote.