Thursday, August 28, 2014

How's College Football Playoff Selection Going To Work?


"Alright, here's how it's going to work..."
 Tonight is opening night for the college football season. Yeah, Georgia State played an FCS team yesterday, but whatever. Tonight's the night that real games start. And this is the first season with the college football "playoff", which will feature a Selection Committee, just like basketball.

A lot of college football writers out there are trying to guess at how the playoff committee will work, and how they will select teams. All in all there seems to be a lot of confusion. And certainly it will be impossible to predict how it will work before it happens. But as there's probably nobody who spends as much time as I do each year studying and understanding how college basketball's Selection Committee operates (here's what I wrote about the 2014 Selection Committee performance), I think I can shed light on a few things.

Let's start at the top, with the biggest issue:

Question: Will the committee take the four best teams?
Short Answer: No, but they'll swear that they did.

Longer Answer:
There are areas where experts in advanced analytics (e.g. sabermetrics) will disagree, but there are other areas that are not particularly controversial. Football teams punt too much, baseball teams bunt too much, and basketball teams really should foul intentionally when the game is tied, the shot clock is off, and the other team has a chance to hold for the final shot.

Another area where there is agreement is that a list of the best teams and the best resumes will not be the same. This is due to the fact that the results of close games are pretty much random chance. Over a large enough sample size, all football or basketball teams will win approximately 50% of their overtime games, so if a team has won four straight overtime games then they've been lucky more than they've been good, and their resume (or list of accomplishments) is above and beyond their actual team quality.

This position generally gets strawmanned by casual sports fans and major sportswriters as not believing that "clutch" play exists or denying that leadership matters or that great players can come up great in great moments. In reality, nobody denies that clutch play exists or that leadership matters or that big players often play at their best in the biggest moments. But the reality is that the total impact of all of those things is just too small to be seen in a sample size as tiny as a single football or basketball season. You can have the most clutch superstar on the planet and still lose four straight games on the final possession. Even in baseball, where the sample size (162 games) is much larger and a one run difference is so much more significant than a one point win in basketball or football, it's basically impossible to see an impact of clutch play on results of games. In 2012, the Baltimore Orioles set a major league record by going 29-9 in 1-run games, and with nearly the identical team the next season they went 20-31. The sample size of 162 games was just too small to draw any meaningful conclusions.

So, with that in mind, we can ask whether the playoff spots should go to the best teams or the teams that have accomplished the most. And I think the answer is that it should be the latter. In the end, even if a game that comes down to the final play is a random coin flip, we want that coin flip to matter. We want to care who wins. That's what makes sports exciting.

And that's what the basketball Selection Committee has always done. They reward the best resumes, rather than the best teams. And that's what we should expect the football version of the committee to do as well.

But here's the catch: The people on the committee do not understand anything I just said.

Confused? Read this twitter conversation I had with David Worlock back in March (start from the top of the page to get the whole discussion). Worlock is the NCAA's Director of Media Coordination. He handles the Mock Selection Committee with the media every year, and is basically as knowledgeable about what goes on in that room as anybody. Yet when I asked him about the topic of best teams or best resumes he responded to me as if I'd asked him which teams had been abducted by aliens:

Now what I said was not at all controversial to anybody who follows the Vegas lines or computer ratings. Utah would have been favored against UMass on a neutral court, as would Tennessee over Iowa State. Yet Worlock could not comprehend this concept. A concept that is absolutely standard in Las Vegas casinos was so foreign to him that it seemed literal insanity. UMass had a better resume than Utah, so therefore they were "better". That is how most of the media and most sports fans view the world as well.

So, when the college football Selection Committee members are interviewed on television, they will tell us (as the basketball committee head does) that they picked the best teams. They will swear up and down that they did. And some angry fans on the internet will point out that their team is rated higher in the Vegas computers than a different team that got into the playoffs. And everybody will scream right past each other because nobody will be speaking the same language.

The four best teams will not be the four best teams selected. Just accept it, and move on. It's for the best.

"No, seriously, UMass is better than Utah. I have no idea what you're talking about."

Question: What official criteria will be used to select the best teams?
Short Answer: Nobody has a goddamn clue

Longer Answer: Now that we've established that the Selection Committee is going to pick the best resumes rather than the best teams, the obvious follow up question is how do you pick the best resumes? And unfortunately, though rather unsurprisingly, nobody can give a straight answer. See this article by Kevin Trahan on some of the mixed messages from the committee itself.

In short, they want to take into account won-loss records, strength of schedule, conference records, conference title game results, head-to-head results, record vs common opponents.... unaware, of course, that all of these things would create different lists of the four top teams. And of course, choosing any of these metrics would divert from choosing the four best teams, which they also say they're going to do. And that doesn't even get into specifics like which strength of schedule metric you want to use, since there are many different choices. And there are plenty of other arbitrary metrics ("Top 25 wins", for example) that will be in play as well. And we haven't even started with conference politics (e.g. Will we allow two SEC teams in? Could we reject the entire SEC one season? Can we deny the Big Ten two straight seasons?). In short, nobody has a goddamn clue.

In the end, while the system is going to be a mish-mash, I expect that there will be pretty high agreement, and that agreement will be very close to the polls. When the human brain is bombarded with too much information, it simplifies that information and searches for the evidence it needs to come to the conclusions it wants.

In the end, if four teams are clear above the rest in the Top 25 polls, expect those four teams to be selected. If spots #4 and #5 are really close in the Top 25 polls, then we could certainly see the Selection Committee disagree with the pollsters. But while there might be some early season disagreements between the pollsters and the committee members, I'd be stunned if the Selection Committee goes far outside the box.

As we've seen over and over again in sports, and also outside the world of sports, big institutions are sensitive to the demands of their consumers. Sports fans come to expect things to be done a certain way, and there will be tremendous pressure on the playoff committee to bend to that will. To use the college basketball analogy, if Tennessee had been seeded higher than Iowa State (even though Tennessee was the higher rated team in basically all of the computer ratings) there would have been literal riots. Literally, cars would have been set on fire in Ames, Iowa. Nobody wants to go against the grain that much.


Question: Will we be able to predict the four teams selected prior to the official announcement? If so, how?
Short Answer: Basically. Look at the polls.

Longer Answer: Once we understand that the Selection Committee is going to be looking at the same mish-mash of information as the pollsters, we understand that the selection process will mirror very closely the polling process.

How does the polling process work? Well, it's an archaic set of arbitrary rules, where teams cannot drop if they win but will always drop if they lose, meaning that a team that loses in overtime on the road at the #1 team in the land is treated worse in the polls than a team that wins in overtime over a 1-10 Sun Belt team, even though the former performance was far superior to the latter. That's just how things go.

Over the larger sample size, a lot of these quirks get washed out, which is why the basketball Selection Committee generally ends up doing a pretty good job by March. In college football, with the shorter season, this will be more difficult.

Expect won-loss records and conference alignment to matter most. If any SEC team goes 13-0, they will be #1 and they will get in. If any teams goes 13-0 from any of the five major conferences they'll get in. Any 11-1 SEC team will get in over any 11-1 team from the Big Ten or ACC. We all know the rules. Strength of schedule should be a very minor factor, although you can be absolutely sure that we'll hear about it constantly in television broadcasts and from mainstream analysts.

One of the biggest problems with the conference obsession is that the mainstream media and casual fans both suck at judging conferences. The SEC's perception as the dominant conference is based on the league being the best from top to bottom over the decade of the 2000s. It wasn't the best every season, but more often than not it was. But the reality is that the era of SEC dominance is over. The Pac-12 has been the best conference in football the last two seasons, and that's a pretty universal and clear perception in every computer rating that I'm aware of. But it will take at least two or three more seasons of Pac-12 dominance before anybody will even be willing to seriously suggest on television that the SEC isn't the strongest league. It just takes a really, really long time for narratives like that to change.

So over the short run, the perception of the conferences will not change. Even if Ohio State ends up with a tougher SOS than South Carolina, there's no way 11-1 OSU will get into the playoffs over 11-1 South Carolina.

Is it possible that the polls will have 11-1 Oklahoma ranked 4th and 11-1 Stanford ranked 5th but the Selection Committee will determine that due to a tougher schedule they're going to take Stanford over Oklahoma? Sure. That's definitely possible. But don't expect any significant deviations from the polls. And so because of that, we'll all be able to predict with fairly high accuracy ahead of time which teams will be selected.


Question: What makes you think you can see the future? Maybe you're wrong about all of this!
Short Answer: True.

Longer Answer: Look, any time you try to predict the future, you can be wrong, so I'm not going to Gregg Doyel myself by declaring as unfit for adult society anybody who disagrees with me. Maybe the Selection Committee will flip the world on its head by, for example, actually selecting the four best teams even if one of them goes 9-3.

But realistically, that's just awfully difficult to see happening. History says that sports leagues give the fans what they want. From instant replay in baseball to goal line technology in soccer to Donald Sterling's suspension and ousting from the Clippers... leagues may drag their feet, and they may come kicking and screaming, but in the end the fans are going to get what they want.

And when a league is trying to institute a new system for determining its champion, the last thing it needs is for the majority of its fan base to not buy in. So they're not going to rock the boat too much.

So don't expect to be shocked by the Selection Committee. Expect to waste a lot of time in September arguing about what will happen if we have six teams that go undefeated. Expect to waste a lot of time in October arguing whether a 12-0 team from the Mountain West should get in over an 11-1 team from the Big Ten. Expect to waste a lot of time in November arguing whether an 11-1 team should automatically get in over an 11-1 team that they beat head-to-head. And expect to waste a lot of time in December arguing whether an 11-2 team that just lost its conference title game should get in over a 10-2 team that finished on a winning streak.

But in the end, we all know how this is going to play out.

No comments: