Smart Use

Economic theory shares some common methodology with engineering in that both formalize design problems as optimization subject to constraints. The framework helps in considering practical problems like the following. An instructor who has used a different course management system in the past is now using the system her campus currently supports. How should she best configure her course site?

In a world where the campus trains the instructor on the software but leaves it to the instructor to make the decisions about course site configuration, the likely solution is to as much as possible recreate the prior course site from the old CMS and where that is not possible kludge the rest. The implicit problem being solved by this approach is to minimize the instructor’s time taken to configure the site while producing something that is tolerably usable by the students.

As with the purchase of new hardware or software that carries a big ticket price, we are often taken with that “sticker” price rather than with the total cost of ownership, in part because we can readily see the former and at best only infer the latter. But clearly, a kludged in site now will create use issues for the students later and usually that means more work for the instructor down the road. Therefore, the initial design of the site really should take into account that “total cost of ownership” approach, to the extent that is possible.

However, the typical instructor, even one who teaches engineering or economics, may be ill equipped to solve that problem, in part because she is unaware of the capabilities and limitations of the technology and in part because she hasn’t solved many like problems and thus lacks familiarity with the requisite type of thinking. We who support these instructors, in contrast, have the experience to do the needed problem solving. The issue then is whether our normal ways of communicating with instructors enable us to communicate the best practice in situ so either the instructor can implement herself or we implement it on her behalf.

Since on my campus we have many large classes that use our course management system and since most of them make heavy reliance on the grade book, let’s focus on that particular area within the CMS. Many of these instructors had previously used either Mallard, our home grown quiz system with some very nice features for doing automated student assessment, or Campus Gradebook, another home grown system, this one a derivative of Plato. Both of these are flat file systems under the hood and for grade book functionality, I believe that can convey an advantage over the database CMS that represent current state of the art, such as the WebCT Vista system that we are deploying. I will elaborate on why below.

Beyond the database versus flat file issue, there is the issue of data representation. Most of the commercial CMS envision a single worksheet view. In the Vista product one can run queries on column characteristics to get a more refined view but that view only endures through the current Vista session. In contrast, one can hide or unhide columns and that action endures till the next time the instructor makes further changes hiding or unhiding columns. Our Campus Gradebook, which we have now retired, allowed a more hierarchical approach. Column data was put into categories – homework, exams, projects, etc. – and row data was put into sections. One could view individual section or all sections pooled, but on the categories one could either view individual categories or aggregates of each category, but one could not view all categories pooled. In other words, there was an asymmetric treatment between columns and rows. Visually, one can scroll up and down easily enough but scrolling across is a pain and hence aggregation is better than keeping track of lots of individual columns.

In an ideal world we’d like a hierarchical grade book as described above, especially in the database driven approach, because each cell in the grade book is its own independent object that must load and hence having a grading with tens of thousands of cells is something to be avoided if at all possible. This is not a concern in small classes. A grade book with 30 students and 30 columns of data is peanuts. But it is big concern in large classes. Furthermore, it is the large classes that are likely to do a lot of quizzing, because it is an efficient mechanism for students assessment in that setting. But then it is possible to get, for example, a grade book with 600 students and 100 columns. Unfortunately, grade books of that scale become almost impossible to manipulate.

We’ve had many large class instructors belly ache about the CMS grade book, but to my knowledge we done little about giving them coping strategies to render the data. At this juncture let me note that how to cope depends critically on whether the instructor is comfortable with Excel or not. If comfortable, the best and most sensible coping strategy is to envision that the grade book is kept by the instructor as a spreadsheet on the instructor’s desktop. Raw data (from online quizzes, for example) can be downloaded into that spreadsheet. All manipulations of the data are done in Excel, and then the instructor uploads into the CMS what the instructor expects the students to view. This requires a sensible approach to backing up the downloaded raw data and the updated spreadsheets, but this is not hard – simply save with the date appended to the file name and then make sure there are duplicates on some other computer.

But for those who don’t feel comfortable keeping their grade book in Excel and therefore want to keep the data and the calculations in the CMS, there is that constraint to deal with that too many cells in the grade book make it perform poorly. What should be done in this case?

If there are TAs and say they have 3 sections under their control with perhaps 90 students in total, then by having one class site per TA perhaps the grade book can have up to 40 columns without becoming unusable. How does one keep from going over that 40 column limit? If there are lots of quizzes, instructors should consider combining them. There is no doubt some pedagogic benefit from giving more shorter assessments that more readily imply the student will get some rapid feedback on his work. But that benefit can be offset by a grade book that becomes unwieldy given too many columns. Thus having fewer, longer quizzes might be a sensible accommodation. A partial alternative is for the instructor to become disciplined and learn to view only a handful of columns at any one time. Most of the instructors I have talked with who have huge grade books insist on viewing all the data in one screen. Other than for self-flagellation, I don’t understand why. Still another alternative, though this requires at least some manipulation with Excel, is to report data in columns like:

In class quiz #8, Sum of in-class quizzes #1 - #7.

That is, give out the most recent info but report prior info only as aggregates. This means the instructor will on a regular basis take down columns and replace with different ones that are more current. And if those quizzes were done online rather than in class it means that after a period the instructor makes those quizzes unavailable to students and then hides those columns in the grade book.

What we who support instruction should not do is allow the instructor with the 600 students and 100 columns of grade book data to wail about how poorly the grade book performs. Lets note that within CMS grades are communicated through the tool – quiz, assignment, etc. – as well as through the grade book. Give them an alternative with which they can manage the constraint and then let’s get on with it. Of course the world and the software we use could be better. But let’s show we can manage in the world we actually live in.

U Phoenix, Online Programs at the U of I, and Extension

Yesterday afternoon and this morning I'm at an online learning retreat for our campus. At dinner last night, I sat next to some colleagues from the Computer Science department. It is one of the top rated programs in its field nationally. They have an online Masters program with about 100 students per year. A question is whether they want to/are able to grow. I was told that a big deal to them is quality of the applicants. They must have good and relevant work experience and have good GPAs as undergrads. This limits their pool of applicants. But it does allow CS to treat its online program as equivalent to the on ground program and to preserve the integrity of the degree.

This is an aggregator Web site of Computer science programs available online. The program at Illinois is not listed, but University of Phoenix is. Indeed, the suggestion that only qualified applicants need apply is nowhere to be found on this site. It does say that programs are either accredited or state licensed. That is the quality assuring mechanism.

After dinner last night President White made some remarks about online education and he asked why Land Grant Universities haven't done more in this area, since it would seem to be a natural continuation of the mission. One simple observation is that the Land Grant mission is manifest mostly via extension programs, which are available to all citizens of the state but are non-degree, while the market that U Phoenix operates in is in the main a degree program market.

But one might make a different distinction, one that President White might harp on. Extension is still fundamentally an agricultural program and run through that college. The Continuing Education possibilities that online learning offers apply mostly to the other 95% of the citizenry, who may very well go through job displacement as a consequence of global flattening. So, I would argue, we should consider whether our historic structures actually block our ability to do current and new ventures in online learning that would open opportunities to people we haven't reached.

This could be our own alumni, or presumably the alumni of peer institutions. Some of them, one might imagine, are facing job pressure from global flattening and one would like to think that we'd think our own alumni would pass the litmus test regarding student quality.

But I don't see us beating down the doors to start a venture of this sort. Is it, perhaps, that we have less to offer people who've already gotten a bachelors degree than one might imagine?

Digitial Repositories versus Virtual "Junk in the Basement"

I am a faculty member and my department, Economics, used to be in the then College of Business Administration. (Now Econ is in Liberal Arts and Sciences and the the former home is called the College of Business.) As a matter of course, CBA gave out Web space to college faculty members and adopted a no quota policy. You can see some of my old stuff still there: the class homepage for my Intermediate microeconomics class from 5 years ago and my personal home page that is from even further back. Most of this was published with FrontPage, but I believe I also used WS_FTP to get content onto the site. I want to make no claims for this stuff in terms of layout or good use of the technology other than that after a period of time, it's still there (though not all the links work).

However, and I realize that those who read this post are looking for ed tech ideas not economics, I'd like to consider my paper "Wage Policies" which is available in pdf format on my personal home page. At the time of the authoring, I thought it was an interesting paper, with a clever idea behind it. I tried submitting it to some of the better Econ Journals, but no luck. Since at that point in my life I was getting involved with ed tech administration, eventually I stopped shopping it around. I note now that neither Google Scholar nor the regular Google search engine can find that paper. So if it is to be archived, the burden is on me (or the College of Business).

Last Thursday night at Educasue, I went to dinner with some of my Frye Institute classmates. One of them is Susan Gibbons from the University of Rochester. We talked about Institutional Repositories. Susan reported that theirs was now going quite well. They had tough going at first and hired an anthropologist to study the research and deposit habits of faculty. One of their "findings" was to make the repository all about "me" from the point of view of the faculty member. In other words, Rochester is using the D-Space application in a way not unlike my homepage on the College of Business site, without the jokes and with more references to scholarly output and with file formats to accomodate the discipline (e.g., audio formats for those in linguistics) . And Susan reported that their "download counter" which gave faculty info about how often their research had been accessed has been very popular.

Susan also reported that many faculty at Rochester had inadequate home pages provided by their department or college and so there was a latent demand for the IR to satisfy this need. Those faculty that did have good support would continue to use that rather than to rely on the IR. My sense is that there is no vetting process for the IR, but perhaps some workflow so that items that are contributed may appear only with a lag. There are still a lot of details about the process that I would like to understand, particularly on what faculty must do with respect to metadata.

If the Rochester experience is replicated elsewhere for the time being we'll end up with a bifurcated approach where some faculty use their department as the host and put up "virtual junk" along with their research on those pages, while other faculty will use the IR. This sounds like a social scientist's dreamworld, to have such a division that invites comparison. Just what is it that we want to know about these spaces? For starters, where do all those paper downloads come from? Are they getting the info from Google or the IR search engine?

DRM

I don't know why but we in educational technology regularly seem to over constrain ourselves in terms of desiderata for services, when something that functions simply and seems to fill a need would seem to nicely. Consider audio reserves. In the old days students students went to the Music Library to sit in a cublicle and listentto recordings that were on some fixed medium. Then some time in the late 90s, someone got a brainstorm and said, wouldn't it be much more convenient for students to stream this content behind some form of authentication to handle both the copyright and the licensing issues, on the one hand, while enhancing student access on the other.

And voila, we have audio eReserves. Now we're seeing students with portable devices that can store audio that is downloaded, and hence further mobility can be given to student, but straight download doesn't pass muster in terms of copyright - the students can redistributed the content very easily. DRM (Digital Rights Management) software applied to this content "can" effectively limite the redistribution. Does this work in practice as well as in theory? I don't know. I'm especially interested in the related support issues if a campus goes this route. If anyone is trying the approach for audio eReserves, please let me know.

Where is Tom Lehrer when you need him?

I spent a good part of the afternoon today talking with folks from various WebCT campuses and all seem quite antsy about the proposed merger. Me, I’ve got other headaches. I get back to my hotel room after have Dinner and a few drinks with buddies from Frye, and there is a message waiting for me from my wife, all stressed out, because she can’t do my kid’s math homework. He is in 6th grade and they are doing something new.

The question at hand: under what conditions can you take four lengths and make them into a quadrilateral? Hmmm. I had an idea and tried to explain it to my wife on the phone. That was a total disaster. So I wrote up the following

https://netfiles.uiuc.edu/l-arvan/Capture1.jpg

But I’m scratching my head asking, can 6th graders articulate this? I couldn’t explain it to my wife, how could my kid explain it to his teacher? I’m all for giving conceptual problems to kids at an early age but if they are really to come up with a general solution to this in a pre-algebra class --- that is asking a lot.

I do think using a Tablet for writing up stuff like this is really cool. If they were cheaper I’d buy one for my kids for Christmas.

More on Deferred Gratification

I’m now in Orlando and finished with my first pre-conference meeting at Educause. It was an interesting session but all these meeting rooms at the convention center are without windows and though spacious they make it hard to concentrate for the entire (3+) hours. How can I complain about that after they gave us an excellent lunch and we did have interesting discussion? Let me get back to that one in a bit.

I didn’t sleep well the night before so last night after the ballgame ended (Congratulations White Sox! Ozzie Guillen has my vote as manager of the decade.) I went right to sleep and slept in till almost eight. Having noted there was a Starbucks across the street from my hotel, I put on some shorts and my tennis shoes and headed over there for a vente coffee. I couldn’t imagine starting the day without it. Last night I had some decaf from the room during the game --- it didn’t quite hit the spot.

I know I’m not alone in this craving for Starbucks (back in Champaign I go to Espresso Royale, which has a near monopoly on upscale coffee places in town). Yesterday evening I drove up from Boca to Orlando and made two stop on the Florida Turnpike. As it turns out Starbucks has one of the concessions at their rest top areas. At each place, the line was enormous, and this was after 6 PM. I think we as a culture have the Starbucks habit. What was once an indulgence has now become a necessity.

By the standards of my parents (I’ve written before about them. They both lied through the Great Depression and their views about work and spending were formed by that.) I am spoiled, no doubt about it. The singular periods where I totally ignored that aspect of my persona was when I went to visit them before my Dad passed away six years ago. On those trips I suspended my needs entirely to accommodate their wants. That way we got along and I understood that. Sure I believe in giving people their own “space” in my interactions with them. But I almost never will deny my own needs in the process. Indeed, I’ll spend some time identifying some of mine so they feel comfortable doing likewise.

What’s the point of this discussion about personal self-indulgence? Here is the question. Suppose for a second we buy the argument that the biggest thing we can teach our kids, whether as instructors in a class or as parents, is to have some patience, do things for the long term reward rather than the immediate joy and more broadly push ourselves in ways that we’d expect their to be benefit down the road, but not immediately. This includes things like studying hard, taking up pastimes that are nourishment for the mind, and being more serious and ambitious about the future. Then we should ask: are we in a position to teach those things to our kids? Have we shown the sacrifice that we want them to develop? Can we teach this if we ourselves are so pampered? And if we don’t pay the price for something, how can we breed anything but the nihilism that seems present in so many of our students?

Those questions seem straightforward enough but perhaps they are too simple. In fact I do put in a lot of hours at work and I think about work fairly regularly in my “down time,” for example during that drive up from Boca. I typically get into the office by 9 AM and am often there till 7 PM or later. So the way I’d refine the above question(s) is as follow: If I’m a confirmed workaholic in some dimensions but I’m self-indulgent and spoiled in other dimensions, can I teach the kids to defer gratification so as to invest in themselves (and society) but tone down the latter? On the parenting side this is very tricky, because the TV, the video games, and the computer games, are all immediate gratification baby sitters that we give to the kids so we can do our own work (or have a few minutes of peace and quiet.) We might have to work less at our jobs so we can work more with our children to teach them this lesson.

But let me take this up in the setting of college instruction. What about in that setting? In days of yore (up to and including spring 2002) I used to teach a large section of intermediate microeconomics and I used a lot of technology in that. I had lectures online (PowerPoint with voice over), quiz questions done in Mallard for students to learn the basics, and problem sets done in a team framework and submitted through WebBoard, with TAs online during the evening to help students with the work. There were a lot of College of Business students in this class and they were quite critical of the course – too much work, more than for courses in their major, stuff was dull or too dry, they had to learn it on their own, I didn’t teach it to them. If anyone thinks that teaching students nowadays to be more serious about their own futures and to work harder to get there will be something easy to achieve, they have another thing coming.

And if individual instructors try this on their own while their colleagues continue with business as usual, well, the chance for this to backfire is enormous and it might take quite a while to establish a reputation for the instructional equivalent of tough love, before any of it will work. So I believe that if this is to work for individual instructors, they have to defer some gratification themselves in their own teaching and possibly how they are perceived as teachers via the course evaluation questionnaires.

This doesn’t bode well. While there might be an equilibrium where all of us instructors are very demanding on our students, the students in turn are serious and put in a lot of effort to do the work, and all are relatively content with the outcome, how do we get there from here? My sense is that large public universities will be the followers in resolving this question (I wish it weren’t true but the economics of instruction suggests that will be the case). Who will be the leaders? Small private universities have the aura of coddling their students. That doesn’t quite mesh with promoting a deferred gratification view.

This is a topic that more of us in learning technology should be talking about. The Net Generation, if anything, is more immediate gratification oriented than its predecessors. (Recall that I don’t usually like to make generational learning type arguments. But there is no doubt that in terms of hours per week exposed to various media that can provide instant gratification, those numbers are higher for my kids than they were for me growing up, and I believe that is typical). So a reasonable argument can be made, I believe, that says to irritate them by working them to the bones rather than accommodating them in their every expressed need. That debate is not happening now. It should be front and center.

On writing, blogging, and podcasting

I’m in Boca Raton now on a stop to visit my mom for a day before heading up to Orlando for Educause. I rented a car in Orlando and drove down here yesterday afternoon. I’ll drive back tonight. On the flight to Orlando I had planned to nap because I was going to do more driving that afternoon but instead I read from my paperback. Normally I read turn pager fiction on flights like that, squished into my seat, uncomfortable and in need of distraction. But I found I had an entire row to myself so I could stretch out and get comfortable. And I had brought the complete short stories of Herman Melville, because I had already started it over the summer and because I had been a little disappointed with the last few page turners. I read a story called Cock-A-Doodle-Doo, one I had never heard of, but one that even in my drowsy state clearly was a masterly piece with glorious sentence construction, a compelling story, a parable with power and emotion, and a social criticism all bound up into one short work.

I’ve wanted to be a writer for some time now, but if Melville is the standard, there is no way; so I’ll content myself with being a blogger. The writer is a craftsman, an artist in the case of Melville. Everywhere in the writing there is evidence of the mastery - the word choice, the imagery that is conveyed, the feeling of uplift that is created in the reader. Bloggers don’t show such mastery. They are in the main reporters. They record what they see, they tie strands of other Web content together via hyperlinks, and they voice their opinion as commentary. To do those functions well, there need not be as much craft in the writing itself. Timeliness of the post matters a lot. Some novelty in perspective is also helpful.

When blogging is truly news reporting it amounts to taking something witnessed over here, perhaps capturing that with photos, audio recording, or video recording and then transferring it back over there to the blog in a way that others can digest. In my blog there is much less news and much more analysis and commentary. For me, generating the prose is not so much a matter of transfer as it is of refining the argument and getting the pieces to fit. It is a good thing for me that sentences have periods and come to an end. The conclusion of a sentence is a chance to do a little check. Am I on track? Can I keep going or do I have to go back and make changes? Further, oftentimes the ideas I have in advance, while certainly necessary to get started are far from complete. The writing forces me to see the argument through to completion. I learn something as a result. Occasionally I’m surprised with the conclusion. I believe the common term is “discovery” – writing to learn means writing is a process that aids the thinking and conversely thinking aids the writing.

Coming into writing this post I had thought the distinction might be that writing is formal while blogging is informal. But that is not it. Melville wrote his short stories for magazines to be read by subscribers during their leisure time. He did not write them to be amassed in some collection, to be studies by scholars a hundred years laters. Melville is definitely a writer but his stories are also informal, disarmingly so. Yet I very much doubt that he dashed one of his stories out in a couple of hours of passionate writing. I suspect he had to layer in his ideas and the writing process he followed somehow had to capture that layering to create an effect that is similar to oil paint on canvas, which can look thick with the layers of different colors and different strokes of the brush. For me, blogging is more of a blurting, a distribution of a first draft that doesn’t intend to get refined. It’s one coat of paint and that’s it.

Now let me skip to something else and then I’ll return to the blogging in a bit. Last night I watched a little bit of the Angels/White Sox game. The third man in the both was Lou Pinella, until recently the manager of the Tampa Bay Devil Rays. When he gave his introductory remarks it was obvious that he was reading from a teleprompter. And it came across poorly because it was so stilted and unnatural. He was a little better during the game and made a few interesting points. But I think he was uncomfortable in that role.

On the same subject but from a different perspective, the last several years we’d have Norm Combs, emeritus professor of History from the Rochester Institute of Technology,
who speaks on the accessibility of online content and who himself is blind. Norm has told me he listens to a lot of books on tape. In his presentations he shows the audience how he uses Jaws, the screen reading program, to hear documents and to navigate around the screen. He has the Jaws program set at a pace that most of us would find incomprehensible. His point is that we read faster than we speak and he has Jaws set to read for him not to speak to him. In his last presentation for us, I was curious to hear Norm say that if people are doing voice recording for the computer they should just talk. They shouldn’t read. I suppose too many of us are like Lou Pinella. We’re not trained to read aloud in the mellifluous voice of the TV commentator (Joe Buck, the play by play guy, never sounds like he is reading from a teleprompter, even if he is.) The listener wants to hear something that sounds natural.

I’ve done a fair amount of voice recording sitting at my own office computer and narrating into the microphone. Mostly I’ve done lectures in microeconomics this way. I’ve found that quite difficult to do. It’s hard to be conscious of the speaking and be reflective at the same time, so the talking is mostly from memory. Unlike with writing there may be an interval of two or three minutes of talking – that is many sentences, perhaps a paragraph or two, where there is no time check whether the argument is on track or not. In the old days when I did this narration to accompany PowerPoint slides I had made, there would be one .wav file for each slide. One could catch a breath and recollect between slides. But during a slide this was almost pure blurting.

So it would seem there are two possibilities with podcasting of lectures and discussions. Either this represents a reading of what was initially a document as text. Or it represents an off the top of the head discussion without much reflection in it. Those who do podcasting as interviews with a dialog and Q&A may be able to get the best of both. One can have a sense of learning when in a dialog that one does not have when delivering a monolog. And yet the speech can sound natural and unlike reading aloud. So podcasting dialog seems like an interesting thing to me.

But I think we’ll be seeing mostly podcasting monolog – either recording of live lecture or recording in the office in front of the computer screen. What do we hope to achieve by that?

I understand the argument for mobility and that giving students access to content on their iPods can be empowering for them. In today’s local paper there is an article about the renewal of inflation and it disproportionately affecting the younger workers. Students on commuting campuses, in particular, may be switching from driving to taking public transportation, as a way to economize on fuel costs. What do those students do while commuting? Giving these students academic content for their iPods would seem to be a real boon. But note that either they have to listent to that content over the din of the bus or training rumbling, or they wear sound cancellation headphones that may be ideal in such an environment but make the students too unaware of their surroundings when riding a bike or walking across a busy street. And let’s not lose sight of the thinking and craft in construction that one will find in such content. Finally, let’s remember that good teaching is mostly not about content delivery.

Drop Date

Today is the last day in our fall semester where students can drop a course without getting permission (from a dean in the college in which the student is enrolled). As one instructor pointed out to me recently, this means the week or so before becomes a period of reckoning for students who perceive themselves on the margin. (And this doesn’t have to be on the pass-fail margin. Student’s who carry a high GPA may want to protect that and drop a course that will have give them a C.) In large classes, in particular, this might create certain communication between instructor and student that could have both a beneficial and a pernicious aspect. On the plus side both likely become more aware of the students standing and progress to date. One the minus side this may result in some haggling about grades that most instructors find unwelcome.

I believe that most people don’t consider campus regulations of this sort and treat them as part of the institutional firmament. But I’m an economist, prone to be anti regulation by nature (really from four years of graduate school nurture and twenty plus years in the profession), and want to consider this along with other campus regulations as to their rationale and their consequence. Also, let me not here that by rationale I don’t mean that campus has various reporting requirements to the state and that some of the campus regulations emerge to fulfill these requirements. I mean, instead, what’s in it for the students and the faculty? With the drop date, for example, one might consider two extremes as alternatives. First, students can never drop a class without getting permission from a dean. Second, students can drop a class even after final exams have been given and the student learns his score on that.

I know that some private colleges do the latter. I don’t think that makes sense here because for students who are not doing well, there is too much of an opportunity for them to spin their wheels before they get some assistance. But I am intrigued by the former. In itself it would substantially reduce the churn that we see at the beginning of the semester. If the “cost” of dropping courses rose students wouldn’t so freely add them earlier. The churn, which is huge and clearly has been facilitated by online registration systems, has no obvious social benefit to me. And from the instructors point of view it is quite pernicious. When does the semester really start, on day one or after day ten?

I know for sure that we in IT, on the both the administrative and academic side of the house, pay a big cost for enabling that churn. In supporting the course management system we have to continually update rosters and, indeed, because there is so much churn the administrative student system has to have enough bandwidth and crunching power to accommodate it.

Does the institution understand the extent of these costs? Who out there now really argues for all the adds and drops. During that early part of the semester they are almost certainly not driven by student performance in the course. What if we took did in fact move the drop date to day one and then took the savings that would produce and put it into advising? Might not that produce a better outcome?

Now let me turn to a related regulation, the grading scheme used by the university. When I start at the U of I in 1980 we had letter grade only. Somewhere in the mid 90s they moved to plus/minus grades, but in an either/or way. Some classes had them, others didn’t. This is insane, especially at a big institution. It creates big cost to sustain. Where is the offsetting benefit? I have a mild preference for letter grade only schemes (less haggling about grades with students) but I’d certainly rather see one scheme applied uniformly than to have instructor choice.

We are in an era where efficiency is paramount and yet I don’t think anyone is looking at all these regulations and asking whether they are serving or hurting the institution. Why not?

More on Agency Issues in Teaching and Learning

Before I get started with my topic let me toot my own horn for a second and note that while everyone is abuzz about the Blackboard and WebCT “merger” I wrote what now seems to be a prescient piece on the CMS industry structure back in May. Maybe that means the rest of my posts aren’t all blather.

* * * * * *

I want to return to the agency framework from yesterday because once one starts to think that way it opens up insights into a bunch of related issues and gives a different way to consider them. Also, I want to make it clear that I’m not picking on the students. There are agency issues with the faculty and agency issues with campus administration, as well.

Let’s begin with course evaluation. It’s part of the institutional culture here and I suspect at many other campuses, but really, it makes no sense whatsoever as it currently is practiced. It relies on students giving their opinions at or near the end of the semester in a manner from which they derive no obvious benefit. In the paper administration their is an element of coercion (the students can’t leave the room until they submit the form to the collector) but in online administration the coercion is absent. The low response rates that almost everyone reports for online administration is an acknowledgement that students don’t want to complete course evaluations. So, first and foremost, the current approach suffers from the problem that student buy in to the process has not been elicited. Yet their opinion is valued nonetheless.

At my campus there are two questions on the evaluation that count in the faculty member promotion and tenure decision (and possibly in their salary decision): (1) Rate the instructor and (2) Rate the course. Outliers on either extreme are singled out, low performers either get reassigned to other courses or if they continue to perform poorly then they are denied tenure or promotion, while outliers on the other extreme become candidates for teaching awards. And perhaps there is some information content in these outlier evaluations, the elicitation of student opinion problem notwithstanding.

But for the rest of us more ordinary souls, there really is scant information in these numbers. Further, it is known that these results are manipulable via the instructor grading scheme and, indeed, grade inflation may be a direct consequence of that. Conversely, grade inflation reduces the information content of these evaluations.

One could evaluate the instruction by looking at student performance, in down the road courses or in the course being taught via artifacts the students produce or via some survey done after the course has been complete where the institution makes clear to the students that their opinion is being sought for the good of the institution and not just to prop up a P&T process that otherwise doesn’t know how to consider teaching. My campus does a senior exit survey of this sort and it is an emblem of this after-the-fact type of evaluation.

If we wanted to continue with course evaluation but in a way that accounted for the student’s preference, we’d deliver two different type of surveys. One would be during the middle of the course (where the instructor could react to it and base changes in course delivery on it). This would ask students about what is working for them and what is not working for them in the course. Then at the end of the course the survey would ask whether the instructor made changes in the course according to the needs identified by the prior evaluation. This process, still not perfect to be sure, does provide an obvious reason for providing the evaluation – to affect the quality of the teaching. (The summative assessment offered at the end of the course acts as an enforcement mechanism for instructors to take seriously the formative assessment offered in mid semester. Students, understanding that, now have a sensible reason for doing the summative assessment at the end.) Some instructors, of course, do try mid semester formative assessment on their own, and that is to their credit. But the institution hasn’t given its endorsement to the practice, so most instructors don’t bother.

Another issue that exemplifies agency issues is grading, particularly the grading of final exams or term papers that are turned in at the end of the semester. In particular, consider the comments the instructors provide in that instance. Do they facilitate learning? Or do they merely rationalize the grade the instructor is giving? My guess is the latter and that one would predict the instructor writes more when giving a poor grade than when giving a good grade. This means, of course, that the instructor views grading as a nasty business rather than as an opportunity to teach and indeed when it is the case that all students have written on the same subject, it is quite dull and trying to continue to pay attention to the writing and find the relative merit of one student’s work over another. I know that many faculty hold machine score-able exams in low esteem, but for finals in particular they have the merit that there is none of this agency issue present (and that the results will be returned to the students in a timely fashion).

For those who don’t want to go that route, consider instead having the instructor evaluation of the work occur before the end of the semester (which means the work must be delivered earlier as well) and then having other students in the class review the work in some manner. There are a variety of ways to do this but in almost all of them the role of instructor changes from one of pronouncing judgment on the students to one where they help the students to learn.

Thus, as with the post yesterday, what is considered good teaching practice should be derivable at some level by a consideration of the various agency problems that crop up in teaching and further that by adoption this agency view one can develop a more realistic way to consider modifications in approach based on evaluative evidence gathered while teaching.

The Principal-Agent Model and Out of Class Work

For the last few years that I did Economic Theory research (in the main this ended as I took up ed tech administration) my work was on various applications of the heavily studied principal-agent model and knowledge of that informed my approach to teaching.

The canonical example of principal and agent is “share cropping” and designing the optimal contract in that environment. The basics of the model are something like this. The share cropper is assumed to be risk averse while the land owner is risk neutral (or risk averse but less so than the share cropper). That implies, all else equal, that in a contract between the parties the land owner should absorb the bulk of the risk and in the process insure the share cropper. The output of the farm depends on two factors. First is the share cropper’s effort. More effort produces greater output but more effort also creates more disutility for the share cropper. Effort needs to be interpreted abstractly and what it means depends on the context. (I’ll say more about that below.) The other factor is a random component, “the weather” or the aggregate of other external factors, that can’t be controlled by the share cropper or anyone else.

If the land owner (and an independent third party who enforces the contract) can observe effort directly, then the optimal contract specifies the effort level where the marginal product of effort equals the marginal disutility (these are the standard conditions that determine the efficient output level) and then the sharecropper receives a flat payment, independent of the actual output, so the land owner gets the entire farm output less that payment to the share cropper, and thereby absorbs the risk. Because economists love jargon, this solution is referred to as “the first best” which really means an ideal solution we won’t see in reality because the information requirements are too high (everyone knows the effort).

The model predicts “the second best” solution where the contract can depend on the farm output, which is observable, but not on effort. In this case, under a flat payment scheme the share cropper would choose zero effort (because effort causes a disutility while the payment does not depend on effort) while under a scheme where the share cropper pays a fixed rent to the landowner and gets to keep the entire farm output less this rent, the effort level would be what it is under the first best solution. But the latter means the share cropper absorbs all the risk, and that is not efficient. The optimal share provides the correct tradeoff between inducing share cropper effort, on the one hand, and having the land owner insure the share cropper against the environmental risk, on the other.

We see elements of this type of solution to the agency problem in sales, where commissions are very important, and in a variety of other cases where one hires an agent, such in the buying or selling of a home. There are some people who like to work on commission, but most people I know would prefer to be salaried and would prefer to have their good effort acknowledged through raises and in non-financial ways that fall under the category “recognition.” In a lot of cases, this happens although the output can be measured, meaning piece rates are possible. The conclusion is that there are other, superior ways to provide incentives and those are compatible with a salary approach.

Most of those ways can be characterized by the idea that the employee has a stake in the work. In the simplest of approaches, the stake is in the form of deferred compensation. A more sophisticated view takes the work itself as potentially rewarding and part of that is how much discretion and initiative the employee is delegated, as well as how much of the work is “play.”

Thus, the employer can create a stake in the job by making the work particularly rewarding. Typically this means the employer must take a longer term view because the employee “doing his own thing” will not pay off immediately, if at all. But the engaged employee is likely to produce more interesting things if given sufficient opportunity to do so.

How do these insights translate into good instructional practice?

First, consider textbooks. How does an instructor and potential adopter of the textbook evaluate one? I know that for intermediate microeconomics books, I look first at the end of the chapter questions? Sadly, in most cases these are dull, unimaginative, an apparent after thought for the author(s). Similarly, many of the ancillary tools for the book are developed not by that authors themselves, who have the biggest stake in the success of the book, but by others. If you ask economists how students learn economics (there are reasons to be suspicious about how economists teach) they’ll invariably tell you by working problems. So in this agency area we seem to be do less well than we might and raising awareness of the issue might encourage the market to produce a better solution.

Next, consider instructors who spend a lot of time preparing their in class presentation. In analogy to the textbook situation, they might find it more effective to instead spend time on the in class “active learning” exercises and out of class assignments that the students do. Further, open ended assignments are intriguing to assign because they have the potential to seriously engage the students. But there is also the problem that students may go astray. So if the instructor does opt for more open ended inquiry, the instructor should be helping students work through those open ended assignments rather than wait for them to be turned in. This work is time consuming. How does the instructor find time for doing it? The answer is by spending less time on presentation and preparing that.

Third, instructors might reconsider what they mandate as required readings for the course. Are the readings interesting on their own account? Will the students retain any of what they’ve read after the course has concluded? Will the readings get the students to consider majoring in that area of study?

All of these ideas fall under the category of a learner centric approach to teaching and none of them are particularly novel when set under that umbrella. So why consider the agency issue at all. Why not simply preach learner centric methods.

The answer is two-fold. One is that instructors, like students, have a basic need to understand “why” one should adopt a particular methodology. What issues does the methodology address? What issues does it create? The agency framework allows the instructor who is contemplating a modification in her teaching to consider the likely consequences.

The second reason, I believe, is the more important one. In general, the first best solution and the second best solution are quite different. An instructor thinking idealistically about instruction may contemplate the first best solution, try to implement a reasonable approximation thereof and then be rudely disappointed by the outcome. The agency framework can help temper the instructor idealism and provide a more realistic way to view the issues at hand.

What about closed content?

This will be a real quickie tonight. I've had some conversations with Librarians here about doing eReserves via Podcasting, but that means folks outside the campus community have to be blocked from accessing the content, for intellectual property reasons, and perhaps some Digital Rights Management is applied to the content so it can not be redistributed, to make the Fair Use case stronger or to appeal to the TEACH Act. The question I want to ask here is whether the end user can be using the same podcatching software in this case. They'd obviously prefer that. Is there any chance that it can become a reality?

More on Open Content

Yesterday I took Merlot.org to task, mildly to be sure, but there is no doubt I did it. Merlot has two ways that contributed content gets reviewed: Users reviews a la Amazon.com book reviews and for some of the content there are professional reviews by hired faculty very much in the mode that peer-reviewed journal articles are refereed. The latter mechanism emerged because some of the founders of the Merlot wanted the creation of learning objects to count for promotion and tenure in the same way that the writing of journal articles counts. Their process was driven by that consideration.

One might have come up with quite a different process if one took as the primary need identifying content that down stream users would put into their courses. Merlot still doesn’t do a good enough job for them. In fields where there is a lot of content to choose from, how do I as a potential user of the content choose? The Amazon.com approach to this offers some information but is it sufficient? Don’t I as a user want to know more about the recommender before I give credence to the review? There are a lot of wackos out there. Why should I rely on their judgment? And, in truth, doesn’t that same criticism apply for the peer reviewed content? Just who is that reviewer and why should I trust that person?

Now let’s step away from the particular issue and ask how people find Web content now. The answer, to me, seems to be that perhaps one starts with a Google search, or one goes to a known “trusted source” meaning a place where interesting and valued content has been found previously. Those are launch places. Then from the trusted sources there are links out to something and one might follow the link from a link, etc.

I want to think a bit more about non-Google trusted sources. And for the moment let’s focus on blogs. So go to Bloglines.com, do a search on your favorite topic (I chose edtech) and then for some of the blogs that come up, click on “Subscribers” and you should see the list of those subscribers who made their subscription public. Then click on a few of them and see what they have as their feeds in the topic you searched. Then a couple of more iterations on the same. I think you’ll find that its seems like there are a few core blogs that many people read and then a bund more out there. Those core blogs end up being the trusted sources I mentioned above. And the commentary that those blogs provide, in my opinion, is similar to the type of commentary that movie reviews provide, expect that the topic need not be movies. The movie review metaphor, however, is helpful here, in my opinion because many people choose what movie to see based on what Roger Ebert says, or in a bygone era what Pauline Kael said.

So how about developing learning object content critics a la movie reviewers? Here’s another area to compare with Merlot. Merlot had reviewers who were subject matter experts (though in some of the science disciplines I believe how expert some of the reviewers were was an issue.) Ebert and Kael are experts in film, not experts in the subject matter of those films. Couldn’t we have generalists who are expert in learning objects that review content across disciplines? We couldn’t if this content creation is to count for promotion and tenure. But in terms of what works for the learner, I think it is more than possible. And, especially, if the bulk of the learning objects are to focus on the first and second year college experience, then this type of review might be much more valuable than disciplinary review. (Of course if the content was fundamentally wrong but was pleasing pedagogically that would be a concern. So in reviewing any single piece of content, it would have to be done in comparison with other content that is already trusted.)

Does anybody do content reviews now? I think that it comes up occasionally en passant in various blogs but I’m not aware of folks who do that as a regular avocation and go out of their way to find new content to review. So the question I want to pose here is whether we in higher ed who want to promote open content should provide incentives for reviews to take place? And if we did, could we make it other than cheer leading and more like film criticism. Typically authors don’t know their peer reviewers in the refereeing process. But everyone knows Roger Ebert. For the film review process to work the reviewer needs the intellectual freedom to say what he thinks.

Consequently, I’m unsure whether a review process of this sort can work. But it seems to me we should explore this approach more before we build yet another repository for online content. In my opinion we need to spend more time considering the social dimensions of the user’s choice and work to improve that. Repositories don’t do anything in that dimension.

Open Content –

The podcasting craze has created a renewed interest in open content that supports instruction. While we in the ed tech business probably think about content too much and so spend too much time on how to make good content (my guys have a nice tutorial in making Web based video content that just came out) it is nonetheless intriguing to talk about how content might be shared. Here I’d really like to separate out “readings” that may be from Journal articles or elsewhere from other types of “learning objects” like screencasts and online quizzes.

One question that arises is for faculty who keep their stuff on their own course Web site, without any password protection rather than in the CMS. That content is already open in some sense. But is it findable? I’m guessing that we have a significant amount of content in this category. For example, consider this site for an introductory course on plant pathology. I did a general Google search for the site and incorrectly used the word “pathology” in the search rather than the word “pathogens,” which is in the course title. It didn’t show up on the first Google page. I’m about 50 – 50 on whether I click through to further pages on the Google search. It depends how hard I’m looking and what shows up on the first page. But I would say that as a student searching, this content could easily be missed.

Of course on the campus Google, it comes up immediately as the first item. So if the instructors only care about making the content known to the local community, they are doing fine, but if they would like their stuff to be more broadly used, one might ask whether they could do some things that would aid the Google placement of their site. More broadly, one wonders whether a campus could do things to make content open content of their instructors more findable.

Here’s another example. Purdue is now offering a service to Podcast lectures and knowing that, I can do a Google search on Boilercast and voila the first link is to a site with lots of content. But if I do a more general search on Podcast lectures, that Purdue page doesn’t come up and instead there a several links to Chronicle pages as well as to Publisher sites. Purdue as an institution has more of an incentive to make that content site well known to the outside world than any individual instructor has. Given the enormous effort they’ve already put in to make this service a go, are there incremental things they can do to make the content more visible?

We know MIT got a lot of mileage in that regard with all the pre-announcements of the Open Courseware Initiative and certainly if an institution can get that type of publicity it will attract a lot of eyeballs to the site. In this case, people will know to search on OCW and then find the site straight away. From there on in, these people will browse, not search, and I think that is helpful because it assists in being able to compare materials and also getting a sense of how much is there at the outset. With the pure search approach, one doesn’t get a good sense of how relevant the links are without going through them all.

So to me, this suggests that a registry would be useful for open content. Merlot plays something of this role but for whatever reasons it hasn’t taken off here and so I’m suspicious about whether a registry where instructors need to enter some info for their content to be listed can really work.

Hence I’m thinking about a hybrid of some sort, where first categories are input, say from OCW or Merlot, and then Google searches are returned on each of those categories. The category labels would then be something of a restricted language and the content creators who really cared about getting their content known would compete by trying to achieve high placement of their stuff in those categories. The other creators would simply be involved en passant.

I don’t know if this cans work, but it is intriguing to me and something I’ll be looking into more.

Radii

One thing that both good managers and good teachers encourage is to reframe questions to encourage a different way of thinking about an issue. When problem solving, the reframing often yields a different path with which to find a solution. So reframing is a good thing to do when stuck on a problem. When doing strategic planning, the reframing offers a different way to consider whether proposed strategies are best given the articulated priorities. In other words, the reframing is a good way to accommodate a diversity of views, especially when those views appear to be in conflict.

I often use the expression “draw a circle” or “draw a bigger circle” as a simple way to reframe because often with IT provision issues a big part of the question is the size of the audience and there is a tendency to draw the circle too small. In teaching, we use the expressions “learner centric” or “teacher centric” but I think we’ve reached the point where those labels are less useful. It is more helpful, I believe, to ask about the size of the audience – the top quartile, the median and above, or the entire class and after having identified the audience then talk about how they are being addressed. Regardless of the situation – teaching, IT administration, thinking about a topic for this blog, you name it, drawing the circle differently can have important implications for the course of action we take.

Let’s consider some examples and let me begin with the following observation. People outside the IT organization have no understanding at all about what IT costs. They look at what’s happening in the home computer market and they abstract, in an imperfect way, from that. Consider storage and backup, for example. A lot of what we pay for in supporting our big systems are costs of redundancy – power supply, hardware failover, data integrity. In other words, self-insurance is a big cost. As I said, the folks outside of IT don’t understand that. But the folks in IT don’t seem to ask, is there a better way to self-insure, at lower cost and with greater reliability?

Here I’m talking outside my area of expertise, but certainly Katrina has brought out the fact that having geographic distribution of the data is probably necessary if you are to do an adequate job of data redundancy. In my IT organization we do a reasonable job of insuring against fire in a building by taking tape backups off site, but against a regional disaster we have no self-insurance.

So why don’t we engage in a grid with like schools to do data archiving elsewhere and in turn we do our share of archiving for our sister institutions? Might we get better insurance that way and do it at lower cost, because then we wouldn’t have to take all the other extreme measures we take to protect the information? I know that in the digital library world the multiple copies approach has a lot of enthusiasts. Why not do that for backing up email, Web publishing, and learning objects in the Course Management System?

Two obvious barriers to me are first privacy and security and second the ability to retrieve the data from elsewhere in case a failure does occur. Those are important issues, to be sure, but are they insurmountable? I’m guessing an important third barrier is that we are egotistical, so we want to be in control. A shared approach of this sort implies less control at the organization level. Control, to the extent it exists, gets passed along to the grid or the consortium that runs the grid.

Ask yourself this question on a personal level. Do you have important data on your home computer that you want to protect? How do you protect those data? Have you considered along with some of your friends providing a way to share your data so you each protect the other? And why doesn’t somebody come up with some software to enable a scheme like this so we all can have better data protection? Why when thinking about this issue do we consider backup an individual task, rather than a social task?

It’s usually easier for me to ask questions about making the circle bigger than to pose analogous quations about making the circle smaller and, in particular, one of the values I bring to the campus IT organization is that I have perspective from outside, given my role as a faculty member. Arguing to make the circle smaller means disregarding people who probably should be valued and so the question becomes how can you in good conscience disregard these folks? And yet, of course, in economics we disregard information all the time so we can come up with a simple enough model to analyze and to make some relevant conclusions. So at least in principle, I’m not antagonistic to the idea to shrink the circle to make the task at hand more manageable. It’s coming up with a good example that is the challenge.

I think I’ve got one – writing. Consider the student writer who does much of the writing to fulfill requirements in the various courses he takes. This writer has many masters to please and to the extent that they are not of all one voice and don’t make overt what in the writing will please them, they create a very uncomfortable situation for the student. Writing in that situation can be unpleasant and writer’s block is a natural consequence, because the writer is stuck on what to do to please the instructor and really there is no way to reframe the task as it is posed to get the student unstuck.

Suppose these instructors went out of their way to get this student off the hook by each of them saying to the student, “Write to please yourself. Don’t worry about how I’ll react to writing. But do be concerned about your own reaction. Are you happy with the writing? Does it say what you want it to say? That’s the goal you are trying to achieve.”

I don’t know too many instructors who actually do this, but let’s say it became the practice. What is the consequence on the writing? One possibility is that students become smug and self-satisfied with their work, so we move away from the writer’s block problem into a world of mediocrity of product that the students expect to be labeled as excellent.

But this is really a delightful turn of events, because now we as teachers have something on which to coach the students and we’ve moved away from the apparent student shirking. So our job becomes one of instilling taste in the students and creating expectations in themselves that they can meet that sense of taste in their own work. We concentrate on taste and we do this by showing what pleases us and why. We model for the students what we want them to do. And then we trust them to respect that and therefore to not be self-satisfied when their work doesn’t reach that level.

Will this type of shrinking of the circle actually work? I don’t know. But I can say that anyone who continues to write after college and does so on a regular basis will almost certainly do it to please themselves. And if others are to appreciate that writing it has to be because that author has a sense of taste that the readers value. So why can’t we get students to think they are writers early on and give them a sense of empowerment in the process?