Sunday, September 20, 2015

There Is No Analytics Debate

The above quote from twitter has long been a favorite of mine. And it's true. The reason most older sportswriters hate sabermetrics/analytics is because they see young, smart guys coming in and overturning the establishment and they relate it to their own industry being overturned by younger, smarter sportswriters on the internet.

In that context, there's no need to debate analytics with sportswriters. When Bob Ryan declares that baseball fans don't care about statistics and that players should only be judged on the eye test, and then (in the same sentence) declares that Miguel Cabrera must be the MVP because of statistics (HR, RBI, BA), we can just laugh at it. We can understand the subtext behind Ryan's ignorant anger.

It's beyond the point that it's worth debating sports columnists.

But my interest was piqued this week when Jerry West said this:

Leave aside the odd argument that "analytical guys" want something other than "the best players" and "a great coach", what's interesting is that a guy who is arguably the most successful general manager in the history of the NBA still views "analytics" as something special and distinct from some "old school eye test", and sees those two things as mutually incompatible:

When I try to visualize this view of "analytics", I'm reminded of the 1994 film "Little Giants", where one of the characters is a nerdy kid named Nubie who builds a computer to come up with plays. He types the situation into the keyboard and the computer spits out a play. The famous play that the computer comes up with to win the big game is "The Annexation Of Puerto Rico". I couldn't find a clean clip online, but a clip below shows the movie spliced between real NFL clips. You can get the gist:

This appears to be how the "old school eye test" people see "analytics". A bunch of people who sit in a windowless room tapping away on a computer and coming up with math equations utterly divorced from what is seen on the field. Dan Shaughnessy, another sportswriting dinosaur, summed up that imagery here:

I hear the argument all the time that there are "analytics people" who believe that only the numbers should be trusted and that "the eye test" has no value. Really? Name one. Name somebody who works for a team, or who has written analytics studies, or who promotes analytics in the media who says this. Find anybody anywhere saying this.

In the real world, analytics is attempting to bring science to sport. And the way experimental science works is that you attempt to quantify what you see and build models, and you then test those models. This means that the statistics and "the eye test" are, in fact, inseparable. Until your models hold up to the eye test, those models are unproven.

What happens when the models/statistics produce a result different from "the eye test"? The anti-analytics strawman is that a bunch of nerds with pocket protectors demand that the numbers be taken as gospel and the scouting be thrown out. In the real world, what happens is that smart people try to figure out why the numbers are out of line with the scouting or old school wisdom, and then they try to determine whether the model is wrong, the data is incomplete, or whether the scouting can be proven to be in error.

The way I put this on twitter when I saw that Jerry West quote was this:
The fact is, in the real world, that there is nobody who is arguing that scouting and "the eye test" not be a part of analysis. There are some people, however, who will not pay any attention to statistics and models that contradict conventional wisdom.

In the real world, there is no analytics debate. There are simply some people who use all of the information available to them, and some people who don't.