Tuesday, October 28, 2014

BREAKING: The CFB Playoff Committee Is Mirroring The AP Poll

This evening, the first college football playoff rankings were put out by the College Football Selection Committee. And figuring it's interesting to see how they mirror or differ from the college basketball version, which I spend a lot of time projecting on this blog, I wanted to discuss just how the rankings worked tonight.

As I projected when I broke this all down two months ago, the playoff rankings basically mirror the Top 25 polls, with only minor deviations. Why did I project this? Well, two reasons:

1) When your selection criteria is impossibly vague, you're inevitably going to just go on feel, and then attempt to justify whatever you decided on with stats. Between W/L record, SOS, head-to-head, resume, eye test and conference standings, there are enough parameters to justify absolutely anything you want.

2) College football, much more so than college basketball, has the "poll culture" engrained. When teams win they move up, when teams lose they drop, and head-to-head matters (but only when the media decides it... since nobody is clamoring for 1-loss Arizona to be ranked ahead of the 1-loss Oregon team that they beat). The college football playoff, more than anything, wants to be accepted by fans. And the absolute worst way they could do that would be "rock the boat" and go outside these accepted norms. College basketball culture is much more accepting of the concept that quality strength of schedule matters, and that Selection Sunday can very much deviate from the Top 25 polls.

So, let's break this down statistically. First, can we demonstrate that the Selection Committee did not care about picking "the best teams"? Sure. I'll use the Sagarin ratings, as I like to do in college basketball for this purpose, because Sagarin has a "PREDICTOR" rating that measures team quality and an ELO_CHESS ranking that purely measures resumes. So below, I have compiled every team that received Top 25 votes (not including FCS North Dakota State and 2-vote Oklahoma State) that had an ELO_CHESS at least five spots apart from its PREDICTOR, and compared them to both the AP Poll rankings and the College Football Playoff rankings:

What do we see? In 11 out of the 12 cases, both the AP Poll and the Playoff Committee chose team resume over team quality. And this is to be expected. When the media says "best teams" they don't actually mean "best teams". They always mean best resumes. I explained this phenomenon here.

But that's not a surprise. The $64,000 question is, does the Playoff Committee mirror the AP Poll more or a true "resume" rating more? Here, I looked at the same group of teams, and selected all of those that had an ELO_CHESS at least five slots apart from their AP ranking. The results are below:

The results look slightly more ambiguous here, in that the Playoff Committee went with the AP Poll only six out of the nine times. But in fact, in all three cases it basically broke a tie by taking the resume. For example, with Notre Dame the committee is four slots off the AP Poll and three slots off Sagarin. With Ohio State they're three slots off the AP Poll and two slots off Sagarin.

On average, among these nine teams, the Playoff Committee was 2.1 slots off the AP Poll and 6.6 slots off the ELO_CHESS. So by a factor of more than three-to-one, the Playoff Committee just mirrored the AP Poll over seriously trying to rate the best resumes.

So, the Playoff Committee was what we thought it was. All the endless debate about how much the committee was going to care about strength of schedule or head-to-head or how good teams looked was pointless. Heck, Jeff Long gave an interview worthy of Kim Jong-un, pretending that he was breaking down blocking schemes and knew when left tackles were out injured in coming up with his rankings. It was all waste of time.

In the end, as we knew all along, the Playoff Committee is just another Top 25 poll. The four spots in the playoffs will be contested by 0 and 1 loss teams from Power 5 leagues along with 2 loss teams from the SEC. That's it. If you are a Notre Dame fan angry that Michigan State, Kansas State and TCU are all ahead of you, don't waste your energy. The odds that all four of those teams will go undefeated the rest of the way is vanishingly small, and all four of those teams will be eliminated with one more loss.

No comments: