Smart Use

Economic theory shares some common methodology with engineering in that both formalize design problems as optimization subject to constraints. The framework helps in considering practical problems like the following. An instructor who has used a different course management system in the past is now using the system her campus currently supports. How should she best configure her course site?

In a world where the campus trains the instructor on the software but leaves it to the instructor to make the decisions about course site configuration, the likely solution is to as much as possible recreate the prior course site from the old CMS and where that is not possible kludge the rest. The implicit problem being solved by this approach is to minimize the instructor’s time taken to configure the site while producing something that is tolerably usable by the students.

As with the purchase of new hardware or software that carries a big ticket price, we are often taken with that “sticker” price rather than with the total cost of ownership, in part because we can readily see the former and at best only infer the latter. But clearly, a kludged in site now will create use issues for the students later and usually that means more work for the instructor down the road. Therefore, the initial design of the site really should take into account that “total cost of ownership” approach, to the extent that is possible.

However, the typical instructor, even one who teaches engineering or economics, may be ill equipped to solve that problem, in part because she is unaware of the capabilities and limitations of the technology and in part because she hasn’t solved many like problems and thus lacks familiarity with the requisite type of thinking. We who support these instructors, in contrast, have the experience to do the needed problem solving. The issue then is whether our normal ways of communicating with instructors enable us to communicate the best practice in situ so either the instructor can implement herself or we implement it on her behalf.

Since on my campus we have many large classes that use our course management system and since most of them make heavy reliance on the grade book, let’s focus on that particular area within the CMS. Many of these instructors had previously used either Mallard, our home grown quiz system with some very nice features for doing automated student assessment, or Campus Gradebook, another home grown system, this one a derivative of Plato. Both of these are flat file systems under the hood and for grade book functionality, I believe that can convey an advantage over the database CMS that represent current state of the art, such as the WebCT Vista system that we are deploying. I will elaborate on why below.

Beyond the database versus flat file issue, there is the issue of data representation. Most of the commercial CMS envision a single worksheet view. In the Vista product one can run queries on column characteristics to get a more refined view but that view only endures through the current Vista session. In contrast, one can hide or unhide columns and that action endures till the next time the instructor makes further changes hiding or unhiding columns. Our Campus Gradebook, which we have now retired, allowed a more hierarchical approach. Column data was put into categories – homework, exams, projects, etc. – and row data was put into sections. One could view individual section or all sections pooled, but on the categories one could either view individual categories or aggregates of each category, but one could not view all categories pooled. In other words, there was an asymmetric treatment between columns and rows. Visually, one can scroll up and down easily enough but scrolling across is a pain and hence aggregation is better than keeping track of lots of individual columns.

In an ideal world we’d like a hierarchical grade book as described above, especially in the database driven approach, because each cell in the grade book is its own independent object that must load and hence having a grading with tens of thousands of cells is something to be avoided if at all possible. This is not a concern in small classes. A grade book with 30 students and 30 columns of data is peanuts. But it is big concern in large classes. Furthermore, it is the large classes that are likely to do a lot of quizzing, because it is an efficient mechanism for students assessment in that setting. But then it is possible to get, for example, a grade book with 600 students and 100 columns. Unfortunately, grade books of that scale become almost impossible to manipulate.

We’ve had many large class instructors belly ache about the CMS grade book, but to my knowledge we done little about giving them coping strategies to render the data. At this juncture let me note that how to cope depends critically on whether the instructor is comfortable with Excel or not. If comfortable, the best and most sensible coping strategy is to envision that the grade book is kept by the instructor as a spreadsheet on the instructor’s desktop. Raw data (from online quizzes, for example) can be downloaded into that spreadsheet. All manipulations of the data are done in Excel, and then the instructor uploads into the CMS what the instructor expects the students to view. This requires a sensible approach to backing up the downloaded raw data and the updated spreadsheets, but this is not hard – simply save with the date appended to the file name and then make sure there are duplicates on some other computer.

But for those who don’t feel comfortable keeping their grade book in Excel and therefore want to keep the data and the calculations in the CMS, there is that constraint to deal with that too many cells in the grade book make it perform poorly. What should be done in this case?

If there are TAs and say they have 3 sections under their control with perhaps 90 students in total, then by having one class site per TA perhaps the grade book can have up to 40 columns without becoming unusable. How does one keep from going over that 40 column limit? If there are lots of quizzes, instructors should consider combining them. There is no doubt some pedagogic benefit from giving more shorter assessments that more readily imply the student will get some rapid feedback on his work. But that benefit can be offset by a grade book that becomes unwieldy given too many columns. Thus having fewer, longer quizzes might be a sensible accommodation. A partial alternative is for the instructor to become disciplined and learn to view only a handful of columns at any one time. Most of the instructors I have talked with who have huge grade books insist on viewing all the data in one screen. Other than for self-flagellation, I don’t understand why. Still another alternative, though this requires at least some manipulation with Excel, is to report data in columns like:

In class quiz #8, Sum of in-class quizzes #1 - #7.

That is, give out the most recent info but report prior info only as aggregates. This means the instructor will on a regular basis take down columns and replace with different ones that are more current. And if those quizzes were done online rather than in class it means that after a period the instructor makes those quizzes unavailable to students and then hides those columns in the grade book.

What we who support instruction should not do is allow the instructor with the 600 students and 100 columns of grade book data to wail about how poorly the grade book performs. Lets note that within CMS grades are communicated through the tool – quiz, assignment, etc. – as well as through the grade book. Give them an alternative with which they can manage the constraint and then let’s get on with it. Of course the world and the software we use could be better. But let’s show we can manage in the world we actually live in.