Tuesday, March 13, 2012

How Well Did The Computers Predict The Field?

If you want to see my full Tournament previews, please click here. Also, as the opening round of the NCAA Tournament is about to start, please join me for the open thread.

But in the meanwhile, I do like to have one of these "How well did the computers predict the field" posts every year. The results are always the same, but the extra data is helpful for convincing people who don't understand the difference between team quality and team resume quality.

Let me get to the numbers first, and then I'll talk about them at the bottom of this post.

RPI

Ten highest rated teams to miss the Tournament (NIT seed given):

43 - Marshall (5)
51 - Oral Roberts (4)
54 - Central Florida (6)
55 - Akron (2)
56 - Middle Tennessee (4)
60 - Miami-Florida (2)
61 - Mississippi (2)
63 - Oregon (3)
65 - Northwestern (4)
66 - Drexel (3)

Ten lowest rated teams to earn an at-large (seed given):
57 - West Virginia (10)
53 - Virginia (10)
52 - South Florida (12)
50 - Texas (11)
49 - NC State (11)
48 - Kansas State (8)
47 - Purdue (10)
45 - BYU (14)
42 - Cincinnati (6)
41 - Xavier (10)

Sagarin ELO_CHESS

Ten highest rated teams to miss the Tournament (NIT seed given):

40 - Northwestern (4)
49 - Oral Roberts (4)
50 - Middle Tennessee (4)
51 - Seton Hall (1)
53 - Mississippi (2)
54 - Minnesota (6)
56 - Drexel (3)
57 - Illinois (-)
58 - Miami-Florida (2)
60 - Mississippi State (4)

Ten lowest rated teams to earn an at-large (seed given):
65 - Iona (14)
59 - Xavier (10)
52 - Colorado State (11)
48 - South Florida (12)
47 - California (12)
46 - NC State (11)
45 - West Virginia (10)
44 - BYU (14)
41 - UConn (9)
39 - Alabama (9)

Sagarin PREDICTOR

Ten highest rated teams to miss the Tournament (NIT seed given):

41 - Miami-Florida (2)
44 - Arizona (1)
46 - Stanford (3)
48 - Seton Hall (1)
50 - Middle Tennessee (4)
51 - UCLA (-)
52 - Minnesota (6)
56 - La Salle (3)
57 - St. Joseph's (2)
58 - Drexel (3)

Ten lowest rated teams to earn an at-large (seed given):
83 - Colorado State (11)
73 - South Florida (12)
67 - Southern Miss (9)
55 - Xavier (10)
53 - San Diego State (6)
49 - Iona (14)
43 - Notre Dame (7)
42 - NC State (11)
39 - BYU (14)
38 - St. Mary's (7)

Pomeroy

Ten highest rated teams to miss the Tournament (NIT seed given):

38 - Miami-Florida (2)
41 - Drexel (3)
47 - Seton Hall (1)
48 - UCLA (-)
49 - Arizona (1)
53 - Stanford (3)
54 - Minnesota (6)
56 - La Salle (3)
58 - Northwestern (4)
60 - Tennessee (1)

Ten lowest rated teams to earn an at-large (seed given):
76 - Colorado State (11)
71 - Southern Miss (9)
66 - South Florida (12)
59 - Xavier (10)
57 - Iona (14)
52 - San Diego State (6)
50 - BYU (14)
44 - NC State (11)
43 - St. Mary's (7)
42 - West Virginia (10)

--------------------------------


How did the computers do?
The results, qualitatively, are the same every year. The Sagarin ELO_CHESS and RPI are measures of resume strength, while the Sagarin PREDICTOR and Pomeroy ratings are measures of team strength. As such, the ELO_CHESS and RPI are always more accurate at projecting the field than the PREDICTOR or Pomeroy. Despite all the media babbling about "the 37 best teams" or "the eye test", that isn't what the Selection Committee wants to know. The Selection Committee wants to know who you beat and who you lost to.

Now that said, the RPI was closer to the Sagarin ELO_CHESS than usual. I'm not sure if that has to do with a change in style by the Selection Committee, or if it's just statistical noise. We'll have to see if next year looks the same way. Historically, with the 64/65 team bracket, we would never have more than one team outside the Sagarin ELO_CHESS Top 50 get an at-large bid. With three added spots to the field I figure that we'll see around three ELO_CHESS 50+ teams get in each season, and that's what we had this year. But we only had four RPI 50+ teams get in.

Which Selection Committee decisions were most inexplicable?
Interestingly enough, the teams that are getting the most media attention for being left out (such as Drexel, Miami, Mississippi State and Seton Hall) are not the teams that had the strongest resume left out. It was actually Northwestern. Obviously Northwestern wasn't considered a team that had a decent chance of getting in on Selection Sunday because they went 8-10 in the Big Ten and went one-and-done in the Big Ten tournament. But the reality is that Northwestern was being punished for being in such a good conference. Put them in the Big East and they go 12-6 and get in. I know that they only went 1-10 against the RPI Top 50, but not all RPI Top 50 teams are the same. There's no shame in losing to Ohio State or Wisconsin or Michigan or Indiana. And there was just a quirk in the Big Ten this year where there weren't a lot of teams with RPIs around 40 and 50. In fact, Northwestern only played three teams all season long with an RPI between 25 and 90. Against those teams? They went 2-1, with home wins against Seton Hall and LSU, and a road loss at Purdue. This is why the blind "RPI Top 50 record" can be wildly deceiving.

Of the teams that got in, obviously Iona had the weakest resume. So in that area, the media perception does align with reality. But other than Iona? Teams like Xavier and West Virginia should have been much closer to the bubble than they actually were. Xavier seemed to get a pass for all of their bad play around the Cincinnati brawl, which seems wrong for multiple reasons. First of all, if you get yourself in a fight, you shouldn't get a "get out of jail free" card with the Selection Committee ignoring those games. And second of all, Xavier played like crap for the first several weeks of being back at full strength.

Of the teams that got in near the bubble, one team that I think got mistreated was BYU. They simply were not one of the weakest bubble teams, and so they should not have been stuck in the play-in game. And putting them in the play-in game messed things up for Marquette, forcing them as a 3 seed to have a play a very good team (either BYU or Iona) in the Round of 64.

Now, as I've said many times, teams are not put into the bracket because of how good they are. If we were doing that then the best teams left out were the University of Miami and Seton Hall, and UCLA actually should have been on the bubble.

Of the teams that got in as at-larges, it's no surprise that Southern Miss, South Florida and Colorado State are rated the weakest. I've been talking all season long about the lucky records in close games for those three teams, and all three rated among the luckiest teams in this post. San Diego State is another team that I've been saying all season long has been wildly underrated overrated.

All that said, I have to say that overall this season the seedings and selections were pretty good. The one team I missed in my bracket was Drexel, though not because Drexel should have been in. I repeatedly protested that Drexel is the exact type of team that the Selection Committee always says that they hate, and that if they had gotten in then Seth Greenberg's head might have just exploded. But Drexel was the media darling and I thought the Selection Committee would cave. The Selection Committee also surprised me by having contingency brackets in place to take into account the Big Ten title game, to give Michigan State a 1 seed.

Obviously the brackets are never going to be perfect. In fact, we can always disagree - there is no "perfect" seeding order. But relative to years past, the Selection Committee overall did a pretty good job this year.

6 comments:

Arnold said...

Ya, I think they did a pretty good job. Can't expect it to be perfect when dealing with NCAA basketball teams.

Anonymous said...

San Diego State wildly underrated?

Jeff said...

Sorry about that. That typo has been corrected.

Anonymous said...

Just looking out for ya! Keep up the good work

Anonymous said...

Not sure why you're always taking shots at the Big East. It wasn't the best conference this year (unlike in previous years), but it was second or third. Northwestern would not have gone 12-6 in the Big East even if they had South Florida's schedule. Also, Northwestern went 2-2 vs teams between 25-90. And if we were to take measure of their record inclusive of RPI 90, it would be 2-4.

In the past when people pointed out how tough the Big East was, you pointed out how bad the bottom of the conference was. Well, the bottom of the Big 10 is horrid and I don't think you've said a word.

Jeff said...

There are only two Big Ten teams outside the Pomeroy Top 100, and the worst is 159th. You have a weird definition of "horrid".