Hot NOT To Rank Teams
Posted Today at 05:08 PM by BasketballPredictions
After last week's talk about ranking teams, I want to follow up with a quick discussion of all of the wrong techniques that people use to rank teams. What is remarkable to me is that basically everybody uses illogical processes to come to their ranking conclusions.
All of these mistakes basically boil down to the same issue: sampling bias. If two teams could play each other in a 15 game series, the winner of a majority of games would most likely be a better team. But since two teams will be lucky to play once when you have 340 teams and ~30 games, this simply isn't possible. Since we have a small sample, we have to acknowledge randomness. Teams do not play to the same ability every single day, so just because a result happens one day doesn't mean the same result will happen the next time those two teams play. That brings us to our first faux-pas:
1. If Team A beats Team B, this does not imply that Team A is better than Team B: On one level, we all understand this. When Oral Roberts beat Kansas last year, nobody ranked Oral Roberts ahead of Kansas. When Mercer beat USC this year, nobody moved Mercer ahead of USC in their rankings. We understand that upsets happen, that one team can get hot and another team can go home, that one team can just get all the lucky bounces. And yet we seem to forget this when the rankings of the teams in question get closer. For example, when Texas beat UCLA this past week, it was assumed that Texas needed to be moved ahead in the rankings. Try going on college basketball message boards and ranking UCLA ahead - people will attack you for not realizing that Texas beat them last week. I mean, Texas won, they have to be better!
What we forget is that the same dynamics that allow Oral Roberts to beat Kansas allow the 8th best team to beat the best team. Now, I'm not arguing that UCLA is definitely better than Texas. Maybe you think that Texas is better. Maybe you think that the game that you saw this past week gave a lot of evidence that Texas was better. And that's fine. But the simple fact that Texas beat them in a single game does not imply, a posteriori, that Texas is better.
If you're not convinced, think of this: If you're saying that Texas is a better team because they won then you are making the logical argument that if two similar teams play that the better team will always win. This brings us to two absurd conclusions: If you know in advance which team is better you can gamble on them and win infinite money, and if those teams played 30 teams the same team would win each time. At the very least, if you think that Texas is better than UCLA then you are saying that in a 15 game series, Texas is going to win atleast 8. Right now they're only up 1-0, so does that imply that the series is over?
2. If Team A beats Team B by more than they beat Team C, this does not imply that Team C is better than Team B: This follows directly from error #1. Again, teams have good days and bad days - sometimes everything rolls in, and sometimes everything rolls out. So, not only can a team like Mercer beat a team like USC every once in a while, but they can also play a team like USC close. This doesn't imply that Mercer is then better than a team that USC blows out. This issue also leads to my third common faux-pas:
3. Don't discount team psychology: When I'm talking about teams having good and bad days, I don't want it to sound like it's total randomness. In fact, most often it's all about a team's psychology. From afar, we can't know everything going on a team's locker room, but there are a few situations that we can predict. One is that teams often play with less intensity when a big opponent lies ahead (the "look ahead game"). Another is that teams often are much more motivated than their opponents when they are big underdogs, because it's simply hard for big-time teams to get motivated for a mediocre opponent. Meanwhile, a bad team can view the game as their most important of the year. Finally, teams often really struggle when the limelight comes onto them and they start feeling national pressure. We saw a lot of this during the past college football season, where teams like South Florida and California played great until people started whispering "National Title." My favorite case of this came in last season's Big East football season. West Virginia was riding high and thinking national title when they headed to Louisville for what was the biggest game in Louisville football history. No surprise, Louisville pulled the upset. That thrust Louisville in the national title spotlight, and the following week they fell at Rutgers in the biggest game in Rutgers history. Finally, the following weekend, Rutgers felt the national spotlight and fell to a mediocre Cincinnati squad.
I don't mean to keep bringing up college football results, but it's just so easy to find so many great examples of team psychology. This past weekend, nobody expected Pittsburgh to knock of West Virginia, and I'm not going to sit here and tell you that I called that one. But I did find myself dumbfounded by something Kirk Herbstreit said the morning before the game. He said that Pitt was a decent team, but that with the National Title on the line there was no way that West Virginia would blow the game. In fact, as any sports psychologist would tell you, West Virginia was most likely to blow the game when the thought of a National Title game wore on them. For a similar reason, this is why the Stanford-USC upset was so much bigger than the Appalachian State-Michigan upset. Depending on which computer ranking you look at, App State is slightly better than Stanford or vice versa. Either way, it's clear that the teams are pretty similar in overall ability. But, App State was a I-AA team playing Michigan in week one. You can guarantee that Michigan players were thinking ahead to their bigger opponents, figuring that they could just show up and win without really trying. Meanwhile, App State had thought about that game for months, as it turned out to be one of the biggest victories in school history. On the other hand, Stanford was a conference opponent. It was USC's 5th game of the season, and USC players can also remember back to when Stanford was good. They were going to come out far more motivated than Michigan was for App State. Which is what made their loss in that game so much more surprising.
Before I get too far off topic, I want to remind people again of the central point I'm trying to get at. Judging teams is difficult, and there is no simple formula. You can't just read the box score and move teams around because of a score. You have to watch the games, take into account player psychology, and try to judge how good teams really are relative to each other. Even if Team A would win 80% of the time against Team B, they'll still lose 20% of the time. So you have to judge if Team A's loss to Team B was the 20% possibility, or the 80% possibility.
I'm sorry that I don't have a perfect formula for you, but it's not my goal here to fundamentally change the way that people think about sports and the world. But if I can keep a few less people from saying "You idiot, how can you have Texas behind UCLA, they beat UCLA!", then I'll have accomplished something positive.
All of these mistakes basically boil down to the same issue: sampling bias. If two teams could play each other in a 15 game series, the winner of a majority of games would most likely be a better team. But since two teams will be lucky to play once when you have 340 teams and ~30 games, this simply isn't possible. Since we have a small sample, we have to acknowledge randomness. Teams do not play to the same ability every single day, so just because a result happens one day doesn't mean the same result will happen the next time those two teams play. That brings us to our first faux-pas:
1. If Team A beats Team B, this does not imply that Team A is better than Team B: On one level, we all understand this. When Oral Roberts beat Kansas last year, nobody ranked Oral Roberts ahead of Kansas. When Mercer beat USC this year, nobody moved Mercer ahead of USC in their rankings. We understand that upsets happen, that one team can get hot and another team can go home, that one team can just get all the lucky bounces. And yet we seem to forget this when the rankings of the teams in question get closer. For example, when Texas beat UCLA this past week, it was assumed that Texas needed to be moved ahead in the rankings. Try going on college basketball message boards and ranking UCLA ahead - people will attack you for not realizing that Texas beat them last week. I mean, Texas won, they have to be better!
What we forget is that the same dynamics that allow Oral Roberts to beat Kansas allow the 8th best team to beat the best team. Now, I'm not arguing that UCLA is definitely better than Texas. Maybe you think that Texas is better. Maybe you think that the game that you saw this past week gave a lot of evidence that Texas was better. And that's fine. But the simple fact that Texas beat them in a single game does not imply, a posteriori, that Texas is better.
If you're not convinced, think of this: If you're saying that Texas is a better team because they won then you are making the logical argument that if two similar teams play that the better team will always win. This brings us to two absurd conclusions: If you know in advance which team is better you can gamble on them and win infinite money, and if those teams played 30 teams the same team would win each time. At the very least, if you think that Texas is better than UCLA then you are saying that in a 15 game series, Texas is going to win atleast 8. Right now they're only up 1-0, so does that imply that the series is over?
2. If Team A beats Team B by more than they beat Team C, this does not imply that Team C is better than Team B: This follows directly from error #1. Again, teams have good days and bad days - sometimes everything rolls in, and sometimes everything rolls out. So, not only can a team like Mercer beat a team like USC every once in a while, but they can also play a team like USC close. This doesn't imply that Mercer is then better than a team that USC blows out. This issue also leads to my third common faux-pas:
3. Don't discount team psychology: When I'm talking about teams having good and bad days, I don't want it to sound like it's total randomness. In fact, most often it's all about a team's psychology. From afar, we can't know everything going on a team's locker room, but there are a few situations that we can predict. One is that teams often play with less intensity when a big opponent lies ahead (the "look ahead game"). Another is that teams often are much more motivated than their opponents when they are big underdogs, because it's simply hard for big-time teams to get motivated for a mediocre opponent. Meanwhile, a bad team can view the game as their most important of the year. Finally, teams often really struggle when the limelight comes onto them and they start feeling national pressure. We saw a lot of this during the past college football season, where teams like South Florida and California played great until people started whispering "National Title." My favorite case of this came in last season's Big East football season. West Virginia was riding high and thinking national title when they headed to Louisville for what was the biggest game in Louisville football history. No surprise, Louisville pulled the upset. That thrust Louisville in the national title spotlight, and the following week they fell at Rutgers in the biggest game in Rutgers history. Finally, the following weekend, Rutgers felt the national spotlight and fell to a mediocre Cincinnati squad.
I don't mean to keep bringing up college football results, but it's just so easy to find so many great examples of team psychology. This past weekend, nobody expected Pittsburgh to knock of West Virginia, and I'm not going to sit here and tell you that I called that one. But I did find myself dumbfounded by something Kirk Herbstreit said the morning before the game. He said that Pitt was a decent team, but that with the National Title on the line there was no way that West Virginia would blow the game. In fact, as any sports psychologist would tell you, West Virginia was most likely to blow the game when the thought of a National Title game wore on them. For a similar reason, this is why the Stanford-USC upset was so much bigger than the Appalachian State-Michigan upset. Depending on which computer ranking you look at, App State is slightly better than Stanford or vice versa. Either way, it's clear that the teams are pretty similar in overall ability. But, App State was a I-AA team playing Michigan in week one. You can guarantee that Michigan players were thinking ahead to their bigger opponents, figuring that they could just show up and win without really trying. Meanwhile, App State had thought about that game for months, as it turned out to be one of the biggest victories in school history. On the other hand, Stanford was a conference opponent. It was USC's 5th game of the season, and USC players can also remember back to when Stanford was good. They were going to come out far more motivated than Michigan was for App State. Which is what made their loss in that game so much more surprising.
Before I get too far off topic, I want to remind people again of the central point I'm trying to get at. Judging teams is difficult, and there is no simple formula. You can't just read the box score and move teams around because of a score. You have to watch the games, take into account player psychology, and try to judge how good teams really are relative to each other. Even if Team A would win 80% of the time against Team B, they'll still lose 20% of the time. So you have to judge if Team A's loss to Team B was the 20% possibility, or the 80% possibility.
I'm sorry that I don't have a perfect formula for you, but it's not my goal here to fundamentally change the way that people think about sports and the world. But if I can keep a few less people from saying "You idiot, how can you have Texas behind UCLA, they beat UCLA!", then I'll have accomplished something positive.
Posts
No comments:
Post a Comment