Failure of Transitivity

Back in the ‘80s when I taught intermediate microeconomics, I would begin with the axiomatic approach to choice which goes as follows.

For a set of alternatives, A, suppose there is a relationship among its members, R, such that

Completeness: For all x, y in A either xRy or yRx. (In other words all pairs of alternatives can be ranked.)

Reflexivity: For all x in A, xRx.

Transitivity: For all x, y, z in A if xRy and yRz then xRz.

In this case R is called a preference ordering over A. To make some more sense of this, when xRy but not yRx, write xPy and say “x is preferred to y,” while if xRy and yRx, write xIy and say “x is indifferent to y.” Thus xRy can be read as “x is preferred or indifferent to y.”

When teaching my students, we’d discuss whether completeness makes sense. Are there cases where two alternatives can’t be ranked? We’d reduce this to situations where the decision maker didn’t have enough information. He or she could rank the alternatives after the fact, when the information was revealed. I’d say we can finesse that issue but for now let’s assume no uncertainty (this is how it is modeled theoretically). Reflexivity caused no trouble. It makes sense that an alternative is indifferent to itself. It would be a problem if it were preferred to itself. Students would invariably call Transitivity “being logical.” It made sense to them, though I’d let them know that upon occasion it failed in experiments where people were tested to see if they act according to their preferences. Mostly in those experiments those subjects where transitivity failed would report they had made a mistake.

When there is a preference ordering and the set of alternatives is finite, there is a choice point, x* defined by x*Ry for all y in A. In other words, x* is at least as good as any other available alternative according to the preferences. It weakly dominates all other alternatives and it itself is undominated. That is the entire theory in a nutshell.

Right now in college football, transitivity is being severely challenged. This season teams that were ranked number 2 routinely lost the following weak and then fell from their lofty perch. And several teams that were number 1, also lost to find their rankings dip. Among the various inconsistencies we’ve seen this season, I particularly like this cycle.

Michigan beat Illinois. Ohio State beat Michigan. But Illinois beat Ohio State.

This is a classic cycle. Based on head to head competition only, it is not possible to rank these three teams. That they are indeed ranked differently reflects their performance against others. This might seem a trifling but the ratings have been remarkably unstable this year because past performance has not been as strong as usual predictor of winning the next game. Contrast the Week 7 BCS and AP poll results where University of South Florida is ranked 2 in both polls and Illinois is out of the top 25 entirely, with the final results, where USF is ranked in the 20s and Illinois has moved quite a bit ahead of them. One might explain the anomaly of USF’s early high ranking by the fact that it was undefeated at the time, but Hawaii was undefeated for the entire season and they are ranked 10^th. Hmmm.

For any situation with three alternatives, when there is a cycle there is no choice among them. Life goes on and a selection has to be made, true enough. But is the selection “rational” according to well established criteria? It can’t be. There is no rational selection. It doesn’t exist.

One sometimes hears the expression, “you have no choice.” But that really means there is one and only one available alternative. Similarly, there is a Hobson’s Choice, where there are two alternatives, the other being the status quo. We have language for those cases, even if the language doesn’t quite match up with axiomatic choice theory.

We don’t really have language for the case where there are cycles. We talk about “making the tough choice” but the issue is typically not that there are cycles, but rather that there are both upside and downside consequences in a binary choice setting.

The paucity of language to describe the situation notwithstanding, I think we face cycles in a lot of settings that are of concern to Learning Technologists. Consider this little scenario, cooked admittedly, but not without realism.

There are three alternatives:

x) Faculty Member uses online materials from Publisher via Publisher hosting and no contract with publisher.

y) Campus writes a contract with Publisher for Publisher hosted content on behalf of all faculty members on Campus.

z) Campus insists that all content possibly involving student grades must reside on Campus hosted Course Management System. Publisher content must be put there, if it is to be utilized.

And then, suppose there are three criteria by which to rank the alternatives:

1. Security
2. Academic Freedom
3. New Partnerships Model

Further suppose that by each individual criterion alone the rankings are straightforward and are of the form:

1) First z, then y, and last x.

2) First x, then z, and last y.

3) First y, then x, and then last z.

Each criterion has a well established ordering of the alternatives but the different criteria disagree as to which is preferred. The question is how to aggregate the criteria. One might impose some further notion, such as fairness, so that none of the criteria are entirely abandoned. One possibility, would be to use Majority Rule. With the above one gets:

zMy, yMx, but xMz,

in other words, a cycle. Technically, there is nothing new here. It is well known that Majority Rule can produce cycles when there are 3 candidates and, indeed, this indirectly is an argument for a two party system. Further, Arrow’s Impossibility Theorem tells us Majority rule is not special this way. There is no rule that produces a satisfactory result for all possible rankings What may be new here, however, is to take this known result and apply to how we make decisions in an IT world.

Standard practice in doing an IT procurement is to first gather requirements prior to choice of product. Each requirement is analogous to the criterion in the above example. Then the various products under consideration are ranked separately by each requirement. Then somehow the results are tallied and aggregated and a choice is made. This can definitely produce cycles, particularly if there is not one product that shines above the rest.

Political scientists have understood the issues with cycles for some time. The power of a committee chair rest in large part from being able to set the agenda and thereby knock off some of the competition early on through preliminary votes. Sometimes we do something similar with IT procurement, where we make certain criteria mandatory. For example, back in late 2002 when we did the Campus RFP for a learning management system, we required the back end to be Unix/Oracle. We did get two bids, one from Blackboard and the other from WebCT. But a potential third bid from Angel was not forthcoming because they were Windows/SQL. At the time, nothing else in the Angel offering could offset that requirement, though in retrospect I think we should have looked at them more seriously, so our faculty and instructional designers could consider their functionality.

The problem with mandatory criteria is to rationalize why some items are in that category while others are only strongly preferred. As it turns out, we based our RFP in part on the RFP written by the Wisconsin System. Ironically, they ultimately went with Desire2Learn, a different Windows/SQL system. Obviously, at some point Wisconsin backed away from the mandatory nature of the Unix/Oracle requirement.

Cycles are becoming more prominent for me now, in my job in the College of Business. I’m confronting the following decision regularly. There is some category of software. That campus might very well provide a flavor in that genre. An alternative possibility is for my College to host a different flavor. A third possibility is to outsource the service. Among the criteria that are important for us in considering these options are Usability and Functionality for students and faculty, Budget, Security, and, for the within College hosting, whether we have the requisite expertise to support the service.

When there are cycles the status quo trumps new entrants. I think we see a lot of that on campus. It translates as inertia. We also see failure to communicate openly on issues of this sort. Is that because other folks recognize the lack of transitivity and are trying to keep the problem under wraps? I don’t know. It sure would be nice to find a Learning Technology initiative we all could embrace.