Tuesday, September 20, 2016

Carson Wentz, And "Moneyball" Confirmation Bias

The NFL is an insular sports league, and it responded as one would expect to hearing that the Cleveland Browns had hired one of the guys from the movie "Moneyball" to be their new President. We were deluged with content from establishment writers asking whether "Moneyball" could work in the NFL (here, here, here, here, and here, among countless others).

As one would expect, the mocking from NFL old timers was swift and brutal. Brian Billick declared it impossible to analyze the NFL with numbers:

Bill Polian dismissed the entire concept of finding an undervalued player:

Leave aside the fact that "Moneyball" is a vague term that nobody has ever really defined, and that nearly every NFL team already has an analytics department, because the narrative was set: We had our new front in the culture war between "old school know-how" and "new school nerds". All we needed was a proxy fight - a specific player or game or statistic that this debate could hinge upon. It appears that we've found one in Carson Wentz.

During the 2016 NFL Draft, the Browns had the second overall pick, and they made the decision to trade down, and to grab a slew of picks (a first, second, and third round pick in 2016 plus a first and second round pick in 2017). This sort of move is an obvious one for an analytics-heavy team to do, as research has shown that trading down in the NFL Draft tends to be beneficial (see here and here). After all, the draft is a crapshoot, and so giving yourself five shots at a good prospect is generally better than putting all of your eggs in one basket. But the media pressed DePodesta on the move, and his safe, corporate answer on resisting the temptation to draft a quarterback just because one is available at #2 turned into "DePodesta thinks Carson Wentz sucks", basically.

And now that Carson Wentz has had a nice two first starts to his career, the final judgment is in: Moneyball has failed in the NFL.

Now, let's not forget how much hype there was for quarterbacks like Vince Young, Mark Sanchez, Robert Griffin III, Johnny Manziel, and numerous other guys early in their careers who later turned into laughingstocks. Let's assume for one moment that all of the Carson Wentz hype ends up being right, and he ends up as a Hall of Famer: Does this mean anything? No.

The NFL Draft is, for the most part, a crapshoot. In fact, everybody who has tried to find "skill" in drafting has failed to find any. Study after study finds that all teams and general managers will, in the long run, draft no better than random chance. In the same way that anybody who sits down at a roulette wheel can win on their first few spins, some teams and general managers will have hot streaks in the Draft, but the roulette wheel will eventually turn against you, as will the Draft. It is why the media crowning general managers because they hit on one or two guys in a Draft is so pointless - it's the logical equivalent of crowning a guy for hitting on "red" three times in a row at a roulette wheel.

So has DePodesta found a way to do slightly better than random chance in the NFL Draft? Maybe? But it will be five or six years minimum before we can even start asking a question like that, and realistically we will need more than a decade of data to answer it with any confidence.

But never mind that, because old school scouts just "knew" that Carson Wentz was a sure thing, until those damned protractors and calculators got in the way. You see, when DePodesta took over the Browns, they brought on a bunch of new scouts (as new team management generally does), and allowed some of their old scouts to have their contracts expire and move on. DePodesta was even nice enough to let these guys go early, to not leave them on as lame ducks through the Draft. This was covered as utter non-news at the time (here and here for examples). When DePodesta traded down, nobody mentioned the scouts being a factor: It was simple analytics orthodoxy to trade down and grab more picks.

Now I don't know if you've heard, but Carson Wentz had two good games, and that has brought out everybody's favorite #AnonymousScout to tell you that he knew all along that Wentz was a sure-fire hit. In fact, DePodesta fired those old scouts because they were so high on that Wentz guy that the "analytics" hated... or something. CBS's Jason La Canfora laid it on thick:

Get that? They fired "experienced", "old school" scouts, general managers, and personnel directors. A bunch of savvy veterans that lived and breathed football. Old-timers who had been through the wars over the years, who had seen everything that there was to see in the sport, and who just knew deep down in their guts how to land a great quarterback. You know, the guys who used first round picks on Johnny Manziel and Brandon Weeden.

Even assuming these scouts are telling what they perceive the truth to be, what they are experiencing is confirmation bias. We are all bombarded with information throughout our day, most of which we cannot hope to process, and so our brains choose to process what they want to process. And so we end up remembering events that fit into narratives we want to believe, and fail to remember events that don't. Scouts all remember those guys that they "knew" were going to be good who ended up being good, and fail to remember the guys they "knew" were going to be good who ended up busting. As I pointed out above, nobody in the NFL has really demonstrated the long term ability to draft better than random chance.

So if Carson Wentz continues to play well, expect to continue seeing the bashing of Paul DePodesta and of "draft analytics". But don't expect to see any bashing of "old school scouting" because everybody passed on Russell Wilson for two rounds or Antonio Brown for five rounds. And don't expect to see any bashing of the New England Patriots because in the 5th round of the 2000 Draft they passed on Tom Brady to grab Jeff Marriott, a lineman out of Missouri who never played a down in a regular season NFL game (what idiots, amirite?). Because those latter two narratives are not narratives that anybody in the press is interested in.

Bashing "Moneyball" in the NFL? The media is very interested in that narrative.

1 comment:

Unknown said...