How To Rank Teams
Posted Today at 03:17 PM by BasketballPredictions
Everybody here has tried to make Top 25 lists. Everybody has criticized the AP or Coaches polls. Why is it so difficult to make a list? We're just ranking the teams from best to worst, aren't we?
Actually, most people don't. And this is the first issue of contention in rankings. Before I really get into my analysis, I want to warn you that I'm going to use a bunch of college football examples. I know that I'm a basketball writer, but it's much easier to analyze football polls since they only play one game per week. In basketball, teams generally play two or three games per week, so it can be harder to see how a team is affected by a single game. If you're still with me, here we go:
College football is abuzz today with BCS talk. Who is Ohio State going to play for the national title? It's going to be LSU, but should it? LSU entered the week ranked behind Georgia, Kansas and Va Tech, and wasn't all too impressive in needing a pick-six to beat a Tennessee team that is a marginal Top 25 squad. If Missouri and West Virginia had both won, LSU would have stayed ranked behind Georgia, Kansas and Virginia Tech. Remember, pollsters almost never move one team ahead of another team (especially late in the season) unless one team loses during the week. But since both of those teams went down, everybody decided that Georgia and Kansas shouldn't be in the national title game since they didn't even qualify for their conference title games. And Virginia Tech shouldn't have ended up in the title game ahead of LSU since they got rocked by LSU earlier in the year. So suddenly LSU jumped all of those teams to move to #2. But if they were supposed to be ahead of all of those teams despite a mediocre performance in the SEC title game, shouldn't they have been ahead of those teams to begin with?
Was LSU worse than Kansas last week, and better than Kansas this week?
Nobody even asked. Nobody even cared. It's about what teams "deserve". We have this archaic system of what a team "deserves" in the rankings. If you win then you don't get passed by anyone behind you. If you lose then you drop. If you beat another team then you should be ranked ahead of them, as long as the rest of your resumes are atleast similar.
But then, since we all intuitively think that these Top 25 polls rank the 25 best teams, we rationalize why these rankings represent the best teams. Yesterday morning I listened to Mark May and Craig James argue over Missouri and Oklahoma. One of them argued that Oklahoma was better because they beat Missouri in their one matchup. The other argued that Missouri was better because they had one loss and Oklahoma had two losses. I forget which argument was made by which "analyst" because I had an aneurysm trying to listen. Both of those arguments are absurd - when Gardner Webb beat Kentucky, does that make Gardner Webb a better team? Sam Houston State is undefeated - does that make them better than 1-loss Michigan State? Those are stupid examples!, you argue. Besides, that's just college football. Basketball analysts don't think that way, right?
I refer you to CBS Sportsline's only columnist devoted entirely to college basetball, Gary Parrish. He had a column a few weeks back, for which I can't figure out how to make a direct link... Go here and scroll down to the November 12th post. He's complaining about the "baffling" basketball polls. Kentucky lost to Gardner Webb and some voters dared to give them some votes. USC lost to Mercer and received 4 votes. Here's the conclusion of Parish's rant:
It's one thing to take a loss when an underdog hits 15 3-pointers or something crazy. I get that. But such wasn't the case with Kentucky or USC. Those two teams were completely dismantled at home, just out-played and out-classed. So get your acts together, you silly writers. I have copies of your ballots and I know who you are. For now, I'm leaving your names out of it because some of you are friends of mine. But this is the last chance. Unless you think Mercer and Gardner-Webb are totally awesome then you cannot still rank USC and Kentucky until they start to look totally awesome. Don't make me go over this again.
So is he right? Since USC got outplayed by Mercer, is it possible to think that they're still a Top 25 team? Of course you can! Even the best team in the country is going to have a bad day once in a while. USC football lost to Stanford. West Virginia football lost to Pitt. If that happened in week one, Parish would have scolded any writer who dared think those were Top 25 teams. But how can anyone argue that those AREN'T two of the 25 best teams in the country?
Basically, Parish is arguing that we ignore everything that we see on the field. We don't make any judgments like "Gee, USC is really missing Daniel Hackett and Davon Jefferson against Mercer". We simply say "USC lost to Mercer" and cease any further intellectual analysis. It is this type of thinking that is "baffling" to me.
Okay, Jeff, so how do you rank teams? How can we just dismiss bad performances as bad games and not include them in the rankings? Certainly it has to be held against USC when they lose to Mercer! So what's your plan?
Well this isn't an easy answer. It's incredibly difficult to rank 340 teams with a small sample size. It's even more difficult in college football where you have 120 teams and 12 games to work with. What I try to do is take in every game for a team. Separate the games into different categories. You have "good" performances, "poor" performances, "typical" performances, and then "Everything that could go right going right" performances, and "The worst game in years" performances. Then, when you try to compare teams, try to compare similar performances. Is Team A better than Team B if both put up "typical" performances. Usually when people are completely out of whack in their rankings it's because they only watched one or two games of a team, and rank Team A's "poor" performance with the best game Team B every played.
If it seems like one team has a high number of "poor" performances then it means you're overrating their "typical" performance. For example, University of Kansas basketball last year. They only had one loss all year against elite teams (against Texas A&M). Their other regular season losses qualify as "poor" performances, with the losses to Oral Roberts, Depaul and Texas Tech. But three "poor" losses in one season has to be held against Kansas. The fact was that the team was fairly inconsistent for an elite team. And this was why I advised people against taking Kansas in the Tournament, since on any given day they could fall on their face.
In the end, all of this boils down to an issue that I'm always harping on. We have to try as best as we can to not suffer from sampling bias. We can't judge teams on a single game. We can't expect that if two teams play that the same result will happen on any given day. Team A may be far better than Team B, and be able to win 9 times out of 10. But sometimes Team B will win, and this won't change the fact that Team A would win the matchup 90% of the time. It requires a lot of perspective to be able to properly compare teams that often play such disparate schedules. And it's impossible to be perfect at this. But at least we should all understand the process. Understand what are valid reasons to rank one team ahead of another, and what aren't. If we don't understand what we're doing wrong then we'll never fix our mistakes.
Actually, most people don't. And this is the first issue of contention in rankings. Before I really get into my analysis, I want to warn you that I'm going to use a bunch of college football examples. I know that I'm a basketball writer, but it's much easier to analyze football polls since they only play one game per week. In basketball, teams generally play two or three games per week, so it can be harder to see how a team is affected by a single game. If you're still with me, here we go:
College football is abuzz today with BCS talk. Who is Ohio State going to play for the national title? It's going to be LSU, but should it? LSU entered the week ranked behind Georgia, Kansas and Va Tech, and wasn't all too impressive in needing a pick-six to beat a Tennessee team that is a marginal Top 25 squad. If Missouri and West Virginia had both won, LSU would have stayed ranked behind Georgia, Kansas and Virginia Tech. Remember, pollsters almost never move one team ahead of another team (especially late in the season) unless one team loses during the week. But since both of those teams went down, everybody decided that Georgia and Kansas shouldn't be in the national title game since they didn't even qualify for their conference title games. And Virginia Tech shouldn't have ended up in the title game ahead of LSU since they got rocked by LSU earlier in the year. So suddenly LSU jumped all of those teams to move to #2. But if they were supposed to be ahead of all of those teams despite a mediocre performance in the SEC title game, shouldn't they have been ahead of those teams to begin with?
Was LSU worse than Kansas last week, and better than Kansas this week?
Nobody even asked. Nobody even cared. It's about what teams "deserve". We have this archaic system of what a team "deserves" in the rankings. If you win then you don't get passed by anyone behind you. If you lose then you drop. If you beat another team then you should be ranked ahead of them, as long as the rest of your resumes are atleast similar.
But then, since we all intuitively think that these Top 25 polls rank the 25 best teams, we rationalize why these rankings represent the best teams. Yesterday morning I listened to Mark May and Craig James argue over Missouri and Oklahoma. One of them argued that Oklahoma was better because they beat Missouri in their one matchup. The other argued that Missouri was better because they had one loss and Oklahoma had two losses. I forget which argument was made by which "analyst" because I had an aneurysm trying to listen. Both of those arguments are absurd - when Gardner Webb beat Kentucky, does that make Gardner Webb a better team? Sam Houston State is undefeated - does that make them better than 1-loss Michigan State? Those are stupid examples!, you argue. Besides, that's just college football. Basketball analysts don't think that way, right?
I refer you to CBS Sportsline's only columnist devoted entirely to college basetball, Gary Parrish. He had a column a few weeks back, for which I can't figure out how to make a direct link... Go here and scroll down to the November 12th post. He's complaining about the "baffling" basketball polls. Kentucky lost to Gardner Webb and some voters dared to give them some votes. USC lost to Mercer and received 4 votes. Here's the conclusion of Parish's rant:
It's one thing to take a loss when an underdog hits 15 3-pointers or something crazy. I get that. But such wasn't the case with Kentucky or USC. Those two teams were completely dismantled at home, just out-played and out-classed. So get your acts together, you silly writers. I have copies of your ballots and I know who you are. For now, I'm leaving your names out of it because some of you are friends of mine. But this is the last chance. Unless you think Mercer and Gardner-Webb are totally awesome then you cannot still rank USC and Kentucky until they start to look totally awesome. Don't make me go over this again.
So is he right? Since USC got outplayed by Mercer, is it possible to think that they're still a Top 25 team? Of course you can! Even the best team in the country is going to have a bad day once in a while. USC football lost to Stanford. West Virginia football lost to Pitt. If that happened in week one, Parish would have scolded any writer who dared think those were Top 25 teams. But how can anyone argue that those AREN'T two of the 25 best teams in the country?
Basically, Parish is arguing that we ignore everything that we see on the field. We don't make any judgments like "Gee, USC is really missing Daniel Hackett and Davon Jefferson against Mercer". We simply say "USC lost to Mercer" and cease any further intellectual analysis. It is this type of thinking that is "baffling" to me.
Okay, Jeff, so how do you rank teams? How can we just dismiss bad performances as bad games and not include them in the rankings? Certainly it has to be held against USC when they lose to Mercer! So what's your plan?
Well this isn't an easy answer. It's incredibly difficult to rank 340 teams with a small sample size. It's even more difficult in college football where you have 120 teams and 12 games to work with. What I try to do is take in every game for a team. Separate the games into different categories. You have "good" performances, "poor" performances, "typical" performances, and then "Everything that could go right going right" performances, and "The worst game in years" performances. Then, when you try to compare teams, try to compare similar performances. Is Team A better than Team B if both put up "typical" performances. Usually when people are completely out of whack in their rankings it's because they only watched one or two games of a team, and rank Team A's "poor" performance with the best game Team B every played.
If it seems like one team has a high number of "poor" performances then it means you're overrating their "typical" performance. For example, University of Kansas basketball last year. They only had one loss all year against elite teams (against Texas A&M). Their other regular season losses qualify as "poor" performances, with the losses to Oral Roberts, Depaul and Texas Tech. But three "poor" losses in one season has to be held against Kansas. The fact was that the team was fairly inconsistent for an elite team. And this was why I advised people against taking Kansas in the Tournament, since on any given day they could fall on their face.
In the end, all of this boils down to an issue that I'm always harping on. We have to try as best as we can to not suffer from sampling bias. We can't judge teams on a single game. We can't expect that if two teams play that the same result will happen on any given day. Team A may be far better than Team B, and be able to win 9 times out of 10. But sometimes Team B will win, and this won't change the fact that Team A would win the matchup 90% of the time. It requires a lot of perspective to be able to properly compare teams that often play such disparate schedules. And it's impossible to be perfect at this. But at least we should all understand the process. Understand what are valid reasons to rank one team ahead of another, and what aren't. If we don't understand what we're doing wrong then we'll never fix our mistakes.
Posts
No comments:
Post a Comment