Tuesday, March 18, 2014

How Well Did The Computers Predict The Field?

This is my annual post where I break down the RPI/Sagarin/Pomeroy numbers of the bubble teams. Below I listed the ten lowest-rated at-large teams and the ten highest-rated non-NCAA Tournament teams. Keep in mind that I am only considering at-large eligible teams, and am not listing automatic bid winners.

My commentary on these numbers is at the bottom of this post.

Note that all of these numbers are as of Monday morning (i.e. they include all of the results up through Selection Sunday but do not include any post-Selection Sunday tournaments... even though a few have already tipped off as I'm typing this).


Ten highest rated teams to miss the Tournament (NIT seed given):

33. Southern Miss (3)
38. Louisiana Tech (5)
49. Missouri (2)
50. Minnesota (1)
53. SMU (1)
54. Florida State (1)
57. Belmont (5)
58. Green Bay (4)
59. Iona (6)
61. Saint Mary's (4)

Ten lowest rated teams to earn an at-large (seed given):
56. Iowa (11)
55. NC State (12)
51. Kansas State (9)
48. Nebraska (11)
47. Xavier (12)
45. Oklahoma State (9)
44. Arizona State (10)
43. Dayton (11)
42. Tennessee (11)
41. Stanford (10)

Sagarin PURE_ELO

Ten highest rated teams to miss the Tournament (NIT seed given):

42. Green Bay (4)
43. Southern Miss (3)
47. SMU (1)
51. Florida State (1)
53. Toledo (6)
57. Louisiana Tech (3)
58. Minnesota (1)
59. Utah (5)
60. St. John's (1)
61. Georgetown (4)

Ten lowest rated teams to earn an at-large (seed given):
63. NC State (12)
56. Tennessee (11)
55. Nebraska (11)
52. Stanford (10)
50. Iowa (11)
49. Arizona State (10)
46. Dayton (11)
45. Oklahoma State (9)
44. Xavier (12)
41. Memphis (8)
39. Colorado (8)


Ten highest rated teams to miss the Tournament (NIT seed given):

32. Louisiana Tech (3)
35. SMU (1)
36. Arkansas (3)
39. Utah (5)
41. Florida State (1)
43. Saint John's (1)
48. West Virginia (5)
49. Maryland (-)
51. Minnesota (1)
53. Clemson (3)

Ten lowest rated teams to earn an at-large (seed given):
73. NC State (12)
60. Saint Joseph's (10)
58. Colorado (8)
55. Nebraska (11)
52. George Washington (9)
50. UMass (6)
47. Providence (11)
46. Xavier (12)
45. Memphis (8)
44. BYU (10)


Ten highest rated teams to miss the Tournament (NIT seed given):

32. SMU (1)
35. Louisiana Tech (3)
36. Utah (5)
38. St. John's (1)
41. Florida State (1)
43. Maryland (-)
51. Clemson (3)
54. Georgetown (4)
56. Illinois (2)
57. Minnesota (1)

Ten lowest rated teams to earn an at-large (seed given):
66. NC State (12)
64. Colorado (8)
53. Dayton (11)
52. UMass (6)
50. BYU (10)
49. Saint Joseph's (10)
48. Nebraska (11)
47. Arizona State (10)
46. George Washington (9)
45. Memphis (8)


How did the computers do?
In general, we see the same results every year. The RPI and Sagarin PURE_ELO, as measures of resume quality, are always pretty close to the final Selection Committee results. The Sagarin PREDICTOR and Pomeroy ratings, as measures of team quality, are pretty far off. As I've said to the point of boredom over the last week, this is the way the brackets are always made. You are rewarded for your resume, not how good you are.

For example, UMass is one of the five or six worst teams in as an at-large bid, but they earned their 6 seed (23rd in RPI, 28th in Sagarin PURE_ELO). And Utah was better than more than a dozen at-large teams, yet they in no way deserved an at-large bid (59th in Sagarin PURE_ELO, 80th in RPI).

Now, what about whether RPI or Sagarin PURE_ELO is more accurate? As always, those two metrics are pretty close. The Sagarin PURE_ELO is a much more accurate measure, but the Selection Committee itself uses the RPI, because they are apparently clueless to the fact that it's a horrible metric that is 20 years out-of-date and easily gamed.

Anyway, historically the Sagarin PURE_ELO has actually been a slightly better predictor of seed on Selection Sunday than the RPI because the Selection Committee has actually proven decent at seeing through badly screwed-up RPIs. And that was probably the case this year again, depending on how you look at it:

A pair of teams in the RPI Top 40 missed out on the Tournament, while none in the Sagarin PURE_ELO Top 40 (this is consistent with past results, where PURE_ELO Top 40 teams very rarely miss out). Ten teams outside the RPI Top 40 got in as at-larges, as well as ten teams outside the Sagarin PURE_ELO, though the Sagarin PURE_ELO did have a team outside the Top 60 get in (NC State) while the RPI didn't. Though I'm not quite sure what to make of that NC State bid, which was just an inexplicable, bad decision by the Selection Committee's part.

Interestingly, the team near the tail end of the at-large pot that was most mis-seeded relative to the RPI was #51 Kansas State earning a 9 seed, which was more in line with their Sagarin PURE_ELO (40th). Kansas State seems like another example of the Selection Committee seeing through a screwed up RPI (Bruce Weber did a bad job of scheduling, and screwed up his team's RPI) to give a proper seed. Another good example not included here is RPI #17 Kentucky earning an 8 seed, which is in line with their Sagarin PURE_ELO (#33).

So as usual, the Selection Committee did a good job of rating teams by resumes and accomplishments rather than how good they are, and they did a pretty good job of seeing through screwed up RPIs like Kansas State and Kentucky. They weren't perfect, but all in all they were pretty good.

Which Selection Committee decisions were most inexplicable?
Like I said above, the Selection Committee actually did a pretty good job this year. The few errors were pretty minor. The biggest error was obviously letting NC State into the field. No matter what logic you want to use on picking teams, there's no way to justify NC State getting in over a team like SMU. And I understand why SMU was left out (the Selection Committee likes to crucify one team very visibly each season that had a horrible non-conference strength of schedule as a warning to future teams), but if you need to have them out for that reason then take a team like Green Bay or Florida State that was more deserving than NC State.

As far as seedings, there really were no teams significantly off where they should be. The worst, honestly, was New Mexico being a 7 seed when they really deserved something like a 4 (14th in Sagarin PURE_ELO, 12th in RPI). But there's no other team that you can really argue is off by more than a seed line or two.

So while the Selection Committee does a terrible job of lying when they go on tv and say they're taking "the 36 best teams" when they're clearly not, and it's obviously disconcerting that the people on the Selection Committee don't seem to comprehend the fact that "best resumes" and "best teams" are not two perfectly overlapping lists, they appear by accident to get the brackets done pretty well each year. And this year was a very good job. Don't let it be said I don't compliment the Selection Committee when they deserve it.

1 comment:

Unique Computer Cases said...
This comment has been removed by a blog administrator.