NFL Draft 2022: Finding a better way to value draft picks

In less than two weeks, Kansas City Chiefs general manager Brett Veach will begin what many are calling the most crucial draft of his career. He will enter the 2022 NFL Draft with a dozen picks at his disposal — not to mention more than a few substantial needs on the team’s roster.

At present, most observers expect Veach will not use every one of their picks to select a player. With a well-earned reputation for aggressiveness over his first four drafts, he is expected to make trades — either before the draft to acquire a veteran player or during the draft to move up (or down) the draft board.

So now is an excellent time to take a detailed look at how draft picks are currently valued — and how the models now in place can be improved.

Current draft valuation models

There are three commonly-used draft pick valuation models. The most famous (and, as far as we know, the most frequently used) is the Jimmy Johnson chart created by the Hall of Fame head coach while he was with the Dallas Cowboys in the early 1990s — which has become so ubiquitous that it is often called “The Chart.” Another is the one developed by Rich Hill of our sister SBNation site Pat’s Pulpit — which is (unsurprisingly) known as “the Rich Hill chart.” Then there is “the Fitzgerald-Spielberger chart,” created by (you guessed it!) Jason Fitzgerald of AboutThe Cap and Brad Spielberger of Pro Football Focus

So let’s compare them.

Please note: the Rich Hill model is based on a 1,000-point scale, rather than the 3,000-point scale used by the other models. So for this comparison, it has been converted to a 3,000-point scale.

As we see here, the Jimmy Johnson model (hereafter “JJ”) and the Rich Hill model (“RH”) are nearly identical. That shouldn’t be a surprise, because the RH model is said to be based on historical draft trades. Since the JJ model has been widely used for more than two decades (and was said to be loosely based on historical trade data), the RH model simply confirms that the JJ chart is the current standard.

And if you ever run across someone who says, “I actually prefer to use the data-based Rich Hill chart because it’s much better than the seat-of-the-pants Jimmy Johnson chart,” you’ll now be able to tell them something they don’t (apparently) know.

We also see that the FS model is substantially different than the other two. And it should be because it’s based on an entirely different data set: salaries paid to NFL players.

What to like (and dislike) about each model

The JJ chart has one big thing going for it: everyone uses it. Whether you like it or hate it, you have to pay attention to it. As the old saying goes, it’s the gorilla in the room.

The 3,000-point scale is also nice because it allows nicely-detailed values ​​without using annoying decimal points.

But it has many problems. One is that it only runs out to the 224th pick. That made it obsolete almost immediately upon its formulation — because, in 1994, the NFL started mixing in a round’s worth of compensatory picks after the third through seventh rounds; these so called comp picks are meant to compensate for team losses in the free-agent market.

An even bigger problem with the JJ chart is that it places virtually no value on picks after the sixth round. Pick 193 has a value of 14 points, while pick 224 is valued at just two points. We’ll all agree that players taken in the seventh round are less likely to have a significant NFL impact than those taken with the first overall pick. But to suggest that the first pick is more than 200 times more valuable than the 193rd selection (or even worse, 1,500 times more valuable than the 224th pick) simply staggers the imagination.

why? Because according to the NCAA, more than 16,000 college football players were eligible to declare themselves for the 2019 NFL Draft. Just 254 were selected that spring. Even if you think that the talent pool is more likely to consist only of the roughly 6,500 Division I players who were available that year, this still means that the college players drafted in 2019 were all likely to be above the 95th percentile of the talent pool

While it is way more than likely that a seventh-round pick will not “stick” in the NFL, a handful of them do precisely that every season. As the NCAA data suggests, it’s probably not these late round picks don’t have the talent to play. It more likely has to do with whether they are drafted into a good situation for them — and whether they’re able to find an opportunity to get on the field. So while the pick used to select them may not be worth a lot, it is certainly worth something

Kansas City Chiefs v Buffalo Bills

Taken by the Chiefs with the final pick of the 2009 NFL Draft, Ryan Succop played in Kansas City for five seasons — and is now beginning his 14th season in the league.
Photo by Tom Szczerbowski/Getty Images

That brings us to the FS model, which places a high value on late-round picks. This is primarily because of a built-in bias of its salary-based formula: it is based on the second contracts of players drafted between 2011 and 2015. This means that draft busts — that is, players who don’t earn a second contract after their rookie deals conclude — aren’t really represented. In addition, every NFL player is paid a minimum salary based on their experience. These two characteristics of the FS model tend to overvalue the late round picks.

As its authors note, this does remove a lot of variance from their model. But you don’t need me to tell you that even first-round picks can (and do) become busts. Variance absolutely exists in the draft market. If a draft chart is to be accurate, it must take this into account. And it must also factor in the on-field contributions that each drafted player provides to their new team — something that none of these models consider.

Still, the FS model does give greater value to picks in the draft’s middle rounds. Anecdotal evidence would suggest that this is a significant problem with the JJ chart.

Doug Drinen’s AV model

2008, Pro Football Reference founder Doug Drinen did some pioneering work on a draft value chart based on the approximate value (AV) metric he had created.

This statistic places a single numerical value (always a whole number) on any player’s season, at any position, from any year going back to 1950. While it reflects standout performances (quarterback Tom Brady is the career leader in AV, leading a top 10 that includes Peyton Manning, Brett Favre, Jerry Rice, Ray Lewis and Reggie White), it also credits players who get playing time in secondary roles — for example, as reserve offensive linemen or special-teams players. An AV of 4 in a given season is considered average.

AV isn’t perfect. As I’ve noted in previous posts referencing the statistic, it has the word “approximate” right there in its name. But the other word is “value” — which made it perfect for Drinen’s purposes.

He looked at the AV of every NFL player drafted over the 20 seasons from 1980 through 1999. (His idea was that by 2008, most of the players drafted in 1999 would have finished their careers). In order to have the data more closely reflect only the AV provided to the team that drafted each player, Drinen discounted AV earned after each player’s fourth year. (It’s important to note that his dataset included 14 seasons that took place before free agency started in 1993).

To account for variance, Drinen presented his data in a multi-column table representing five tiers. I’ve charted them to give you a clear visual perspective of what they represented, adding the appropriate logarithmic trendlines to better represent the raw data.

The “Top 3” data is just what it says it is: the average of the top 3 players in adjusted career AV who were selected at each pick of the draft; it represents the maximum value a team could expect with a given pick. Then the successive tiers broke out the variance. Drenen gave an example of a team holding the 22nd pick.

[The team] could conceivably get a 124 draft value player [the “Top 3” tier] if they get really lucky. They have about a 20% chance [the 80th percentile] of getting a 52 or better, a 40% chance [the 60th percentile] of getting a 39 or better, a 60% chance of getting a 26 or better, and an 80% chance of getting a 14 or better.

Note that Drinen was taking those numbers directly from the raw values ​​in his tables, represented by the dashed lines in the chart. The trendline would indicate that the “really lucky” drafting team would be more likely to get a player who would give them something closer to a career AV of 104 — not 124.

Drinen’s approach had other flaws. One was that in order to increase the sample size (and “smooth out” the data) for each draft pick, he averaged the data for five picks on either side of each selection. For example, the value for the 22nd pick was the average of the data from the 17th through the 27th picks. This created a problem for the first (and last) five picks of the draft.

Furthermore, in his effort to account for multiple levels of variance, Drinen made the chart difficult to comprehend. He never explained which set of data should be used to value draft picks — although the 80th or 60th percentile data is probably what he had in mind. So his idea never caught on.

Still… he was on the right track.

Plotting the logarithmic curve for Drinen’s 60th-percentile data (and adjusting it for a 3000-point scale) alongside the other standard draft charts, we see that it falls between the FS model and the JJ model. Like the FS model, it gives greater value to picks made in the second and third rounds (which is likely to be a flaw of the JJ model) — but unfortunately, it fails when it reaches the final rounds; some seventh-round picks have a value less than zero.

It has one big advantage: the data is based on the performance of players on the field. Somehow, Drinen just went about it the wrong way.

So can we fix that approach? That’s what we’ll cover next week in the second part of this series — in which we’ll create an AV-based draft value chart that makes sense

Leave a Comment