The mantra - data driven decision making - has had a big influence on us, some in ways that are obvious to see, others perhaps far less clear. One new thing I've become aware of is that several students in my class whom I would judge are not top notch analytically are nonetheless majoring in econometrics or double majoring in economics and statistics. This, almost surely, is being done because the choice of major is driven by the perception of where the good jobs will be after graduation. Knowing how to manipulate data so its truths can be revealed is an important skill.
But I keep coming back to this Koopmans piece, Measurement without Theory, as a critique of the "data will tell all" view. As I was trained to be a theorist, I think you need to start with questions that you want the data to inform about. Those questions then take the shape of a model, where parameters can be estimated with the data and hypotheses can be tested by the data. Indeed, in the bits of empirical work I did as an administrator, first in the SCALE project, then more than a decade later as the CIO for the College of Business, the questions I was trying to answer drove the inquiry as well as how the data was amassed. No fancy statistical techniques were employed, yet I knew enough about the data to be able to get at useful answers to my questions. I'm not saying that you don't need knowledge of econometrics or statistics to understand what's going on. I'm saying that you need theory too, which is the lesson I drew when Freakonomics was the rage.
There is yet another reason why data won't tell all. This happens when the information needed to answer the fundamental questions is not present, but is potentially attainable with some effort in data collection. That is the issue I want to address in this piece. In my previous post I argued that additional instructional resources needed to be put into the first-year experience. An immediate rebuttal might be something like this - we're a public university and can't afford to teach first-year classes in a more labor intensive way. Anyone with a traditional view of the public university - meaning geezers like me who remember back to 39 years ago - will likely embrace the rebuttal because exclusively large lecture classes in the first year seemed a staple of how things were done. In turn, it was how the cost of instruction was kept down.
But things have changed, a lot, since 39 years ago. For one, as I argued in this post called, The business and ethical dilemmas of undergraduate education at public R1's, tuition has been hyper-inflationary over essentially this entire time interval and now constitutes a major source of revenue for the university. Purely on the matter of making things transparent to the "customer" (I hate to think of students as customers when they are in my class, but surely the university needs to consider them this way as they or their families pay tuition) there is a need to communicate what they are getting for what they pay. How much of their tuition goes for direct expenditure on instruction, particularly the instruction they are getting in their classes? What would a good number look like? I don't know but my prior is that it should be around 50%. Having the right data would inform that view.
Another way that things have changed is the path students take to the degree. It's now common for students to take the first two years elsewhere, either at a community college or at some other university, and then transfer in, mainly as juniors. Such students have already completed most of their general education requirements and are probably taking fewer large lecture classes. If the old model had the first two years of college subsidizing the last two years of college, but the transfer students don't pay this subsidy, why should the students who do start at the university pay this subsidy?
I don't want this to be a metaphysics discussion. I want it to be practical. Let's begin with this question. Can we determine a number that measures the size of the subsidy? Here's a second question. Might some of the subsidy be going elsewhere, e.g., to doctoral education or to research or service? This is a trickier question to answer. I don't want to get bogged down by it here, so let me use that question to ask yet a different one. Might there be other ways where the subsidy manifests? For example, might some majors subsidize others or some colleges subsidize others? This you could answer in a fairly straightforward way, with the type of information I'm arguing we need to have.
Now let me talk a bit about methodology. Instructional units are determined by the number of students enrolled in the class times the credit hours the course offers. The class I'm teaching now currently has 36 students registered and the course offers 3 credit hours, so the course is generating 108 IUs at present. Yet I have two phantom students on my roster (students who stopped doing the course work quite early in the semester and who stopped coming to class). That sort of thing is probably hard to measure from one class to the next, but what is readily measurable are the number of late drops and the number of students who fail the course. So one can get a more refined view of IUs, by excluding those students.
Likewise, enrollments typically vary over the semester. When I did that SCALE project, the data I got from DMI (the Division of Management Information, which curates institutional data) had 10-day enrollments and final enrollments for all undergraduate sections. Students can't add a class after day 10 without permission of the instructor, so day 10 is referred to as the add date. The drop date is much later. (I think it is day 40 - 8 work weeks into the semester - but I'm doing that off the top of my head so that number should be verified.) My preference would be to have looks at enrollments on day 1, day 10, day 40, and final enrollments. The reasons for this are many. Here are a few of them.
Students engage in some gaming of the course registration process. They can't tell which classes are easy and which are hard, so they use the first 10 days as a way to sample the classes and their instructors. Students also have strong time-of-day preferences for when to take classes, but many courses are at capacity early so they register for something else, at a less desirable time, hoping to get into a more preferred offering at a better time. Then, students may under perform in class so consider dropping it after they learn their scores on the first midterm. What, then, is the right time during the semester to measure IUs from a theoretical perspective? I don't have a good answer to that question. Instead, my preference would be have several different views to consider and then see if they matter much in doing the expenditure per IU calculation.
Let me talk now about the expenditure side of things. In my case this is remarkably easy. I am under contract to teach the one course I am teaching this fall. So my pay in that contract is the appropriate expenditure number. With full-time faculty, it is somewhat harder. Teaching load matters as does the fraction of the total time devoted to instruction. In the model we used when I first started at Illinois, the typical teaching load in the Economics Department was 4 courses per year (fall and spring, summer teaching was extra) or two courses per semester, with the typical allocation that one course was undergraduate and the other was graduate. Also, the typical time allocation we would state is 50% research, 40% teaching, and 10% service. If this actually were still the case you'd take the instructor's (9-month) salary, multiply by 40% to get the part of salary devoted to teaching, and then divide that number by 4 to get the part of salary on a per course basis.
While I know there is now an official requirement for time reporting, I strongly doubt that most faculty actually track their time allocation. Further, there is a conceptual issue with measuring cogitation. If you're thinking of your model while driving to work or while doing the dishes (something I normally did when I was doing economics research) does that time count? In any event, if you were able to get an an hours per week measure of the activities, it might produce quite a different number as to the salary per course number. So the numbers one gets won't be precise, for sure. Yet it would still be interesting to have those numbers, to get a look at salary per course.
A different issue arises in computing such expenditure for discussion sessions run by TAs. This is whether their tuition waiver should count in their pay or not. There is some incentive for the campus to count the tuition waiver, as it will raise the expenditure per course number. But let's face it. The main reason for using TAs to staff these discussion sections, rather than rely on full time instructors, is because it is cheaper that way. So I'd like to see the numbers without the tuition waiver included. And by the philosophy I've articulated above, it really would be good to have both views. I don't want to argue for a single number. A vector is better than a scalar here. We'd get a better sense of what's happening that way.
Let me close by making one more point. One of the adjustments that students have made to the current system is to take more credit hours per semester and therefore to devote less time in any one course. This semester I have one student who self-reported that he is taking 24 credit hours. Several students reported taking 18 credit hours, and among them most seemed to me not that strong analytically. This gaming, either by having a double major, or by trying to accelerate the time to graduation, is tyranny of the extensive margin and ends up crowding out deeper learning on the intensive margin, which needs students to put in more time. Some of the student stress we're seeing is a consequence of this sort of reaction, with the students themselves not perceiving they should want to find in their studies a form of self-expression. This gives a measurable reason for wanting to see expenditure on instruction per IU. The hope is that if we reallocated resources in a way that shows the students we understand their dilemma, we might change the living hell they are currently experiencing into a reasonably nurturing experience. That should be our goal.
No comments:
Post a Comment