Monday, March 18, 2013

How Well Did The Computers Predict The Field?

My full set of NCAA Tournament previews is in the process of being written. The parts I'm done with are posted here, which is where the rest of them will be posted when they're completed. I want to briefly take a break from those to do my annual "how well did the computers predict the field" post.

I have some thoughts on the computers this year, but those will all be at the bottom of this post. Let's get to the numbers first. Please note that teams listed below are only potential at-large teams - it's meaningless to talk about teams with automatic bids to the NCAA Tournament.

RPI

Ten highest rated teams to miss the Tournament (NIT seed given):

31. Southern Miss (1)
54. Louisiana Tech (5)
56. UMass (2)
57. Kentucky (1)
59. Tennessee (2)
60. Alabama (1)
63. BYU (3)
64. Detroit (6)
65. Denver (3)
66. Stephen F Austin (5)

Ten lowest rated teams to earn an at-large (seed given):
53. California (12)
52. Villanova (9)
50. Cincinnati (10)
46. La Salle (13)
45. Iowa State (10)
44. Boise State (13)
43. Pittsburgh (8)
41. Temple (9)
40. Illinois (7)
39. Oklahoma (10)
Sagarin ELO_SCORE

Ten highest rated teams to miss the Tournament (NIT seed given):

29. Iowa (3)
32. Baylor (2)
40. Virginia (1)
44. Kentucky (1)
48. Maryland (2)
49. Stanford (4)
50. Tennessee (2)
55. Alabama (1)
58. Purdue (-)
61. Southern Miss (1)

Ten lowest rated teams to earn an at-large (seed given):
59. Middle Tennessee (11)
57. La Salle (13)
56. California (12)
54. Boise State (13)
52. Temple (9)
51. Colorado (10)
47. Wichita State (9)
46. Illinois (7)
42. UCLA (6)
41. Oklahoma (10)

Sagarin PREDICTOR

Ten highest rated teams to miss the Tournament (NIT seed given):

29 - Kentucky (1)
30 - Baylor (2)
32 - Iowa (3)
35 - Virginia (1)
47 - Stanford (4)
49 - Maryland (2)
52 - Denver (3)
53 - Tennessee (2)
55 - Alabama (1)
56 - Southern Miss (1)

Ten lowest rated teams to earn an at-large (seed given):
63 - Boise State (13)
61 - Temple (9)
58 - California (12)
57 - La Salle (13)
54 - Butler (6)
51 - Colorado (10)
45 - Oklahoma (10)
44 - Middle Tennessee (11)
43 - Villanova (9)
42 - UCLA (6)

Pomeroy

Ten highest rated teams to miss the Tournament (NIT seed given):

27 - Virginia (1)
29 - Iowa (3)
38 - Denver (3)
41 - Baylor (2)
47 - Kentucky (1)
48 - Stanford (4)
55 - Maryland (2)
56 - Southern Miss (1)
60 - Stony Brook (7)
61 - Alabama (1)

Ten lowest rated teams to earn an at-large (seed given):
68 - Temple (9)
58 - California (12)
57 - La Salle (13)
52 - Butler (6)
51 - Oklahoma (10)
50 - Colorado (10)
49 - Boise State (13)
45 - Villanova (9)
44 - UCLA (6)
42 - Illinois (7)


--------------------------------


How did the computers do?
We had a change this year, which was that Sagarin replaced his ELO_CHESS with his ELO_SCORE. The logic is that the ELO_SCORE has the same basic ELO principles, but does take scores into account a little bit. And this was never going to be as accurate of a measure of team resumes as the ELO_CHESS was, but I was very curious to see how much the final bracket would come close to it versus the RPI. And the answer is, the RPI was a better projector of seed than the ELO_SCORE. In fact, the RPI was a better projector of seed than any time in recent memory.

At the same time that the RPI was so accurate, teams that were really good but had resumes featuring holes (Virginia, Iowa, Kentucky, Baylor, etc) were all left out. In other words, everything you hear about television about "picking the 37 best teams" and "the eye test" is even more of a lie this year than usual. The Selection Committee really could not have cared less about how good teams were. If they were, then taking out Temple and replacing them with Iowa would have been a no brainer - something which no bracketologist anywhere considered doing. Even the biggest Iowa homers don't think that should have happened.

Hopefully Sagarin goes back to his ELO_CHESS next year, because it's disconcerting when the RPI is becoming more powerful instead of less. We know that the RPI sucks, and any mid-major coach who isn't designing their schedule to manipulate the RPI should be fired - it's so easy to do and so valuable. This isn't where we should be trending in the year 2013, with so much information at hand.

Which Selection Committee decisions were most inexplicable?
The Middle Tennessee pick, though not obvious above, is far and away the most inexplicable. The Selection Committee always weighs big wins and bad losses as more valuable than having neither, and so the correct decision would have been putting in Virginia over Middle Tennessee. Clearly, though, the Selection Committee was spooked by the massive gap in RPI between the two teams.

Right after the brackets were announced, the chair of the Selection Committee was interviewed by CBS, and his answer on Middle Tennessee was perhaps the most nonsensical I have ever heard. Those interviews are always a joke, and you expect the typical lies that they actually pay attention to Pomeroy or that they're "trying to pick the best 37 teams". In fact, as soon as you start talking strength of schedule or "who did you beat" or "best resumes" then by definition you're not taking the 37 best teams. If you want the 37 best teams then you rank the 37 best teams regardless of any of that other stuff. It is logically impossible to put in the 37 best teams and the 37 best resumes and the teams that played really well on the road and had a tough non-conference schedule. Each of those groups of 37 teams is different.

Yet even on top of that, the argument for Middle Tennessee was that they were "road warriors", and that winning games on the road was important for the Selection Committee. The only problem with this is that the best team Middle Tennessee beat on the road this year was Central Florida, and Middle Tennessee was undefeated at home. So every single loss they had came on the road or on a neutral court. That's not what you usually think about when talking about "road warriors".

If we take out considering of "best teams" and just look at resumes, it's clear that the most inexplicable seeding decision was dropping Oregon to a 12 seed. No matter which way you look at it, they should have been higher. And it's ironic since Oregon was the single most overrated team in the human polls all season long that they ended up being the team most under-seeded by the Selection Committee.

As for teams left out of the bracket, the team that should feel most snubbed is one that get zero media attention - Southern Miss. If we're going to throw a bone to Middle Tennessee, why doesn't Southern Miss get one? Their resume was stronger. They were not considered by the media a serious contender because of only three RPI Top 100 wins, and that certainly seemed like a strong argument since three wins would have tied the fewest for any at-large team ever. But then Middle Tennessee got in with one. So go figure.

Anyway, the moral of the story is that we have to hope Jeff Sagarin brings back the ELO_CHESS next season, and we have to hope that the Selection Committee goes back to a more reasonable measure of resumes than the RPI, or that they at least have a better sense of encouraging beating good teams, instead of just playing them.

3 comments:

Anonymous said...

I just found your blog this year and I am tremendously impressed. I agree with most of what you put up here. However, (always a "but" right?) I don't think I am with you on the SMiss over MTSU thing. "victories over top 100" just seems too darn arbitrary. Who is to say it is not harder to beat one 47 than 2 96s and a 65? I know your point was HOW MTSU was explained, but even if we accept that SM bad loss (RPI 208) is the same as MTSU's (164), we still have to weight the wins. 1 power conf sub 50 win in 4 tries not that bad in a weak bubble year.

DMoore said...

I think there is a pretty clear explanation for why MTSU was picked, but if that's the case, I don't understand why it isn't the explanation we're being given. The biggest thing Middle Tennessee has going for it is how they scheduled in the non-conference. In the part of their schedule they had control over, they really tried to play lots of good teams.

I think it makes perfect sense for the committee to reward teams for choosing to play a tough schedule. The biggest thing that supports this theory is that if you look at the teams that were close but not selected, a great many of them had Very weak non-conference schedules.

But if that's what they were intentionally doing, I would think they'd want to trumpet it loud and clear. Instead, we get obvious evasions when the committee is asked to explain who got in and who got left out.

Jeff said...

Yeah, DMoore, I get that. Although what good is a tough schedule if you don't beat any of them? Also, Middle Tennessee's non-conference schedule wasn't quite as tough as the RPI makes it out to be. Other computer ratings have it near 40th, instead of 9th.