Fanbogs - College Football Weblogs

August 14, 2004

Ranking the BCS components

Ok, so sometimes I get bored and write programs for lack of anything else to do. Last december was one of those times, and I wrote programs to measure the performance of the rankings used in the BCS. It was my hope to show that the computer rankings, especially my favorite the Colley matrix, were compairable to the human polls. Fortunately, the NYTimes poll did show itself inferior to most of the others, and I could show that the Margin of Victory ristriction imposted on the computer rankings reduces their effectiveness.

The rankings used were the final rankings used by the BCS in their Bowl arrangement. I planned to measure 3 benchmarks:

Accuracy: The number of games correctly picked over the season.
Prediction: The number of bowl games they got right.
Stablitity: How much the final rankings changed due to the bowls.

I never got to stability. Mostly because I could see the writing on the wall, and didn't like the results.

Also, when compairing to the human polls, I only used games featuring teams recieving points in both polls. ( I also extrapolated the human rankings down to the last team recieving votes.)

If a computer ranking offered a BCS ranking and a preffered, excluded, ranking, I benchmarked the excluded ranking as well. The one used by the BCS has an O post-fix to its name, for official. The excluded one has an A post-fix to its name, for Actual.

Ok, so here are the results:

NameAccuracyAll GamesPredictionAll Bowls
MasseyA44-19545-12711-420-9
AP Poll45-18NA11-4NA
Coaches Poll43-20NA11-4NA
SagarinO46-17553-1199-617-12
Wolfe44-19545-1299-618-11
SagarinA41-22536-13610-520-9
MasseyO41-22546-12610-518-11
Billingsley40-23547-12510-519-10
**NYTimes**42-21550-1227-817-12
Colley41-22546-1268-715-14
Anderson42-21540-1328-713-16


** Not in the 2004 BCS formula

Conclusion:

The humans are better at ranking teams than the computers, in general. Massey and Sagarin are closest to matching the human polls in effectiveness. The MOV restriction on Massey's algorithm is detremental to the BCS formula.

My favorite ranking, the Colley Matrix, is not sophisticated enough to compete with the human polls. I may have to become more of a Wolfe fan this year.

(PS... in 2 weeks (Aug 30th) I plan to begin my weekly MWC notes...)

 

Comments:

  1. Fanblogs Author Kevin Donahue said:

    posted on August 14, 2004 12:18 PM — linkabuse?



    I don't get it, Ben. How are the computer polls "predicting" games?

  2. Fanblogs Author David Stein said:

    posted on August 14, 2004 12:18 PM — 4.65.110.210 — linkabuse?



    Therein lays the problem: for without the computers, we return to the nonsense of the human polls and the continuous favoritism for certain teams. All we would see in the top five would be: Oklahoma, Florida State, Miami, Nebraska and Notre Dame. Because, if you remember, the majority of these media polls are anti the West Coast, thus producing a biased product. However, you cannot return to the computer polls, because they too have "malfunctions." I.e. 2001 and 2003. So, I then must pose this question: What is the point? For, doesn’t the BCS defeat the purpose of College Football? College Football is not about championships and trophies, but about beating your rival. There was a time where USC vs. UCLA, Miami vs. Ohio St, and Army vs. Navy...was more important than winning a trophy. There was a time when the Rose Bowl was between the Pac-10 and the Big-10. But that is not more. The media has tried to make College Football into a professional event. However, they fail to realize, that these players are still kids and they are still developing athletically. Money isn't everything, especially when it ruins something as good as College Football.

  3. Matt Yeager said:

    posted on August 14, 2004 4:23 PM — 4.12.29.71 — linkabuse?



    Wow, so humans are actually more accurate than computers?

    Told you so. Regardless... I actually have no real opinions on this, except that maybe we need to scrap the BCS, or eliminate the computers. Or at least decrease their importance. But that's been my opinion for about, oh, six years.

  4. Fanblogs Author Ben Prather said:

    posted on August 15, 2004 3:20 AM — 69.73.43.133 — linkabuse?



    Kevin:
    I assumed that if a computer ranking sais one team is better than another it is predicting that the higher ranking would win.

    The accuracy measure is how well the rankings used to determine the BCS standing used to determine bowl matchups did at getting the regular season right. A better result means that, after the season, they got more regular season games right.

    The prediction measure is how well the indicated formula did at indicating who would win a bowl game. A better result means the ranking, before the bowls, was better at ranking the winner of bowls above the teams they beat.

    I like David's point. I wish it were still true. Utah vs. BYU is still a game I will count as most important on Utah's schedule.

    Today there is a push, especially by the media to crown a single champion. It is true that every other college sport crowns such a champion. A play-off is not in reach for college football. The question is how should the champion be determined? I am too drunk right now to answer that question.

    Matt:
    Despite EVERYTHING I stand for, right now Human polls are better than computers. Sagaring, Massey and Wolfe are close. ( Billingsley is closer than my rankings indicate because they weight later games more, hurting them above... my old favorite, Colley, can't say this.)

    They have decreased the computers from 50% to 33%.

    I also noted that the MOV restriction is bogus.

    I also indicate my distast for my own results. A large part of my favortism towards Colley and the computers in general ( and the MOV restriction) is that the inaccuracies they reflect favor the Mid-Majors. ( Follow the Colley Matrix this year if you want proof )

    As a final note,
    If a CPU cycle is compairable to a neuron firing in its computational strength, then the PC's on our desks are somewhere near an ant in computainoal power. Deep Blue is closer to a mouse. In 30 years we might get to some of the more intellegent primates, and in 50 years we sould get close to the human brain.

    This is all based on the current exponential growth rate of processing power and computer memory, which has lasted over 60 years, and continues strong today. With quanum computers on the horizon, and molecular level designs taking center stage in research institutions, this trend should continue for quite some time.

  5. Fanblogs Author David Stein said:

    posted on August 15, 2004 12:25 PM — 4.65.110.210 — linkabuse?



    Actually there is a way to create a playoff system: Eliminate the non-conference games. That would open up about 3-4 extra games. The only problem with this idea is that the Universities would be loosing T.V. revenue. However, if they are able to put their greed aside, this is a viable solution.

  6. Matt Yeager said:

    posted on August 15, 2004 4:50 PM — 4.12.29.71 — linkabuse?



    That is a truly silly idea.

    Eliminate non-conference games? Couldn't be done. You're not going to ask Clemson/SoCar, USC/ND, and countless others to just give up their rivarly games, are you? Even lowering them to one wouldn't work, since, say, Idaho has two rivals, Washington State and Boise State. You'd also more or less eliminate I-A/I-AA games, which are HUGE boosts to I-AA schools. No way.

    A playoff simply isn't going to happen, because while you can have, say, basketball players play two games in three days, you can't have anything remotely similar for football. Sorry.

  7. Fanblogs Author Kevin Donahue said:

    posted on August 15, 2004 6:18 PM — linkabuse?



    A playoff is possible, but very difficult to manage with more than 8 teams. You could incorporate the current bowl system. There's a billion writeups on this across the net, including a couple dozen here at Fanblogs.

  8. Fanblogs Author TigerEducated said:

    posted on August 18, 2004 12:16 AM — 4.230.216.85 — linkabuse?



    An anti-west coast bias...The West Coast being such a one horse burg and all...I mean, it can't possibly compare to thriving media centers like the ones found in the SEC (Fayetteville, Oxford, Baton Rouge, Gainesville, etc...)...

    Los Angeles, I mean...Los Angeles...Of COURSE it gets a bad wrap...I mean, look at the sports teams there...The Lakers...the Dodgers...What a lack of coverage we see...

    I mean, Kobe & Shaq...Now there was a story that went totally under the radar of most of the rest of the nation due to that terribly anti-West Coast bias, man. I've also recently heard that Kobe is facing a court case of some sort. Maybe you guys closer to the action on the Left Coast can clue us in as to just what's going on out there.

    National Sport Shows that travel and do remotes at myriad locations nationwide like Jim Rome that emanate from Los Angeles just don't do justice to the thriving sports scene in California...

    Think about the NFL, for example. I think that, with a wing and a prayer, Los Angeles and the surrounding area could possibly support a franchise. Call me crazy, I know, but their population and TV market are gradully creeping up with that of such thriving areas like New Orleans, Green Bay, and Jacksonville...

    (Please excuse my laughter as I type this, and my heavy use of sarcastic condescension in this post to help my left coast friends realize that the entire reason their precious USC won a title last year was because of the behemoth media machine that rolled into action out there for them)

Please note that all comments are subject to the Fanblogs Comment Policy.