ncaa rankings

I will divert my attentions in a few weeks toward the possible religion I actually follow, that being college basketball. In advance of this, I looked back over the various computer methods of ranking teams, picking games, and so forth. I discovered something which has been obvious for several years to analysts and serious fans of the game. They use the system that is fundamentally most useless to evaluate teams during the bracket setup: RPI. The RPI ratings system has become completely useless now that it has been re-weighted with the discovery of road victories as so slim and valuable. At no point does the RPI exceed even the completely subjective polls in it's ability to 'predict' games, meaning it has little to no value as an evaluational tool. Basically what the RPI does is measure the strength of scheduling that a team does. But it does not effectively measure the strength of each team. Which makes it fundamentally flawed because that is in a sense what it was intended to do, but not what it has been designed to do.

The result is that teams have learned to manipulate the system by scheduling aggressively. That may be by design, and is a fair and useful outcome. But to then use this to justify an entire rankings system which is then a component of the overall evaluational profile of various teams when potentially large sums of money are on the line for the bids that are presented or not presented as a result is not a justifiable situation.

Some examples. Drake is currently 6th in RPI. Granted they are 20-1 (2-1 vs top 50). More reasonable systems, even the subjective polls, have them somewhere between 15 and 30. I can accept that a team which has one loss but has played a mediocre schedule is still a good team, but not a top flight program. I saw another example looking over mock bracket projections. Complaints were logged because Vanderbilt was placed as a '10' seed. Which is roughly where I would put them if it were up to me (possibly lower still). RPI has them as the 11th team in the country, far cry from 40th. Nothing against a relatively decent academic program doing well in big time athletics (Stanford/Duke/UofM for example). But to conceive that they're doing that well when it's plainly obvious watching them that they're a good team, but not a dangerous one, indicates a serious flaw in the calculations or the assumptions used to make them.

Reversing the scales. Wisconsin is listed as 20th on RPI. I have them in the top 10. Kansas St is 32nd, I have them in the top 15. West Virginia is at 48, they're in the mid-20s on my scale. There are points of agreement. RPI's emphasis on road warrior types is useful, as poor quality road teams tend to get trounced in the neutral tournament style games later on. It puts a point on who a team plays. That's important. It's also important to play well or even win against who they play. And RPI doesn't evaluate this at all. It's true that having so much on the line tied to how much a team wins by in it's games has some ramifications that would be unsportsmanlike and fraught with equally bad side effects (gambling, teams scheduling mediocre teams to beef up scoring margins or teams sandbagging everyone for gambling purposes, different styles of play, etc). But it does seem that there must be a better fit curve out there to use than the current one which seems to track to nowhere in particular and doesn't do much good as an aide to making that last day of bracket building any easier.