Tuesday, March 16, 2010

How Well Did The Computers Predict The Field?

I'll go back to bracket previews in my next post, but I always like to have a post on how the computer predicted the field. Obviously the Sagarin ELO_CHESS will always be the best predictor of the field, but the question is how good, and also whether the RPI is gaining or losing importance in picking teams. Here are the numbers:


RPI
Ten highest rated teams to miss the Tournament (NIT seed given):
40 - Rhode Island (2)
43 - Wichita State (3)
45 - UAB (2)
47 - Kent State (4)
53 - Memphis (3)
54 - Dayton (3)
55 - Mississippi State (1)
58 - William & Mary (5)
59 - Virginia Tech (1)
60 - Seton Hall (4)

Ten lowest rated teams to earn an at-large (seed given):
62 - Minnesota (11)
56 - Florida (10)
50 - Marquette (6)
49 - Notre Dame (6)
48 - UNLV (8)
44 - Missouri (10)
42 - Florida State (9)
39 - Wake Forest (9)
38 - UTEP (12)
37 - Louisville (9)


Sagarin ELO_CHESS
Ten highest rated teams to miss the Tournament (NIT seed given):
36 - Virginia Tech (1)
40 - Seton Hall (4)
47 - Rhode Island (2)
50 - Cincinnati (2)
52 - Mississippi (2)
53 - Mississippi State (1)
54 - South Florida (3)
55 - St. John's (6)
56 - Wichita State (3)
57 - UConn (4)
58 - UAB (2)

Ten lowest rated teams to earn an at-large (seed given):
59 - Minnesota (11)
49 - Utah State (12)
48 - Florida (10)
44 - California (8)
43 - Georgia Tech (10)
42 - Louisville (9)
41 - UNLV (8)
39 - Wake Forest (9)
38 - Clemson (7)
37 - UTEP (12)


Sagarin PREDICTOR

Ten highest rated teams to miss the Tournament (NIT seed given):
37 - Arizona State (1)
38 - Virginia Tech (1)
39 - Mississippi State (1)
42 - Mississippi (2)
43 - Memphis (3)
51 - Miami (Fl) (-)
52 - Dayton (3)
53 - UConn (4)
54 - Illinois (1)
56 - VCU (-)

Ten lowest rated teams to earn an at-large (seed given):
55 - Richmond (7)
50 - Florida (10)
49 - Gonzaga (8)
47 - Wake Forest (9)
46 - Notre Dame (6)
45 - Oklahoma State (7)
41 - New Mexico (3)
40 - UNLV (8)
34 - UTEP (12)
33 - Louisville (9)



Pomeroy
Ten highest rated teams to miss the Tournament (NIT seed given):
31 - Virginia Tech (1)
41 - Mississippi State (1)
42 - Arizona State (1)
45 - Dayton (3)
46 - Miami (Fl) (-)
51 - Memphis (3)
52 - Illinois (1)
53 - VCU (-)
54 - Mississippi (2)
55 - UConn (4)

Ten lowest rated teams to earn an at-large (seed given):
56 - Gonzaga (8)
50 - Wake Forest (9)
49 - Florida (10)
48 - Richmond (7)
47 - New Mexico (3)
44 - Oklahoma State (7)
39 - Louisville (9)
38 - Notre Dame (6)
37 - UNLV (8)
36 - Vanderbilt (4)

--------------------------------


Let's look at the Sagarin ELO_CHESS first, since it's intended to be a measure of overall resume, and it's always the best objective gauge of Tournament bid. What we historically see is that teams with an ELO_CHESS between 40th and 49th have about a 50% chance of making the Tournament, and we rarely see more than one team inside the Top 40 missing the Tournament, and rarely see more than one team outside the Top 50 getting in. This year we had one of each (Virginia Tech and Minnesota, respectively). The Virginia Tech call was a pretty clear one from the Selection Committee: it was a punishment for playing a joke of an out of conference schedule, and also because they didn't defeat the elite teams in their own conference and simply beat up on the bottom feeders (like Providence last season in the Big East). It's the same reason that Penn State was left out last season, when they were the only team in the ELO_CHESS Top 40 to get left out. Penn State went on to win the NIT, so we'll see how motivated Virginia Tech is to repeat that. We're hearing a lot of moaning on television about Virginia Tech, but that's just because they're in the ACC, and Penn State isn't the type of sexy basketball program that anybody on ESPN is going to waste their time defending. But really, they had almost identical resumes. Minnesota is a bizarre case because I can't recall the last time a team with an ELO_CHESS that poor got in. But I think that had to do with two things. First of all, almost all of the other bubble teams stunk it up in their conference tournament, so when the Selection Committee looked for teams that finished well they saw all teams with ELO_CHESS ratings outside the Top 50 (like Mississippi State). And Minnesota was clearly a lot better than their resume (their PREDICTOR is 21st). But the ELO_CHESS also backs up the point that it was a bit bizarre that Seton Hall wasn't considered more of a serious bubble team. I wouldn't have put them in the Field of 65 either, but it's strange to me that so many more people are complaining about Mississippi State than Seton Hall, when Seton Hall is clearly the team with the better resume.


The Sagarin PREDICTOR and Pomeroy ratings are both intended to measure how good a team is. They will never be as good as the Sagarin ELO_CHESS at projecting seeds because they don't measure resumes, but they're the best metrics for figuring out which teams are better or worse than their seed. You'll note a lot of overlap between the PREDICTOR and Pomeroy ratings, as well as a lot of overlap with this list that I put together yesterday. So the best teams to miss the draw? Virginia Tech, Mississippi State and Arizona State. Not a big surprise. It might be surprising to some to see UConn so high on that list, but it's not a big surprise to me when you consider how they played this season. They were an atrocious three point shooting team and it cost them so many close games. They were a lot better than their record. Note that both Miami and VCU made both lists of the ten best teams to miss the field, yet neither even made the NIT. Miami fans (if they care about going to the NIT) should be particularly flummoxed that NC State got invited when Miami is so obviously the better team.

The worst teams in the Tournament? Clearly they're Gonzaga, Wake Forest, Richmond and Florida, in some order. It's not a coincidence that I picked all four of these teams to lose in the first round. A couple of other teams that are clearly much worse than their seed are New Mexico and Notre Dame. I picked New Mexico to the Sweet 16 just because they have an easy draw, but I wouldn't pick them to beat an elite team. And I have Notre Dame going down in the first round.


If you're a regular reader you won't learn much new in this post. You know that the ELO_CHESS is the best objective projector of the Tournament seeds, and of the Field of 65. And you know that the Sagarin PREDICTOR and Pomeroy ratings will always mostly agree about the best and worst teams. But it's always good to see the actual lists to see just which teams the computer ratings think stick out.

8 comments:

Anonymous said...

Are ou posting the rest of your bracket? I didnt see the other side posted

Walt said...

hey Anonymous, maybe go back and re-read the very first sentence of this post. Chill dude.

Anonymous said...

Hey walt I didn't read the post about predicting the brackets. Chill dude

Anonymous said...

So when is the Midwest coming...I love your input but I need to turn my brackets in in the AM!

Jeff said...

I'm working on my Midwest preview right now. It will be posted in an hour or so.

A Different Jeff said...

Give him some time man the midwest is tough to predict!

LamKram said...

I can't see why both the Pomeroy and Sagarin ratings beat up on New Mexico. Yes, UNM is in an easy conference, but they went 7-3 against top 50 teams and 4-0 against top 25 teams, with only one loss to a team out of the top 50 (Oral Roberts). Seams like the 3 seed is justified.

Jeff said...

LamKran, please read the post to understand the difference between how good a team is, and how good their resume is.

The Sagarin ELO_CHESS is a measure of resume, and New Mexico's ELO_CHESS is 10th. So since seeds are awarded on resume (things like the RPI Top 50 records), New Mexico certainly deserved that 3 seed. But Top 50 records don't have anything to do with how good a team actually is.

New Mexico went 9-1 in games decided by five points or less (9-0 in the regular season before losing in the Mountain West tournament). In other words, they were very lucky to have the resume that they have.

What the computer say is that New Mexico deserved their 3 seed, but they're the type of 3 seed you should bet against.