Analysis of NFL teams’ drafting effectiveness from 2000–2020 by comparing player performance relative to draft spot
by Jason Pauley
I have tried in the past to use career AV (Approximate Value) as a metric when analyzing the draft. The problem I have always struggled with is being able to compare recently drafted players to players drafted many years earlier, because of the difference in AV related to years in the league. I have often left active players out of the analysis because of that and focused the analysis only on players out of the league. (see AV notes at the end of the post)
Here is my solution to normalize draft picks, draft spots, team draft performance, trends etc. for the purpose of comparisons and analysis:
I have taken each player’s Career Weighted AV and divided it by the sum of all AV from that player’s draft to come up with a new metric, AV Share. Here are two examples:
Tom Brady was drafted in 2000. He currently has a career weighted AV of 180. The sum of all players from his draft has a career AV of 4,568. His AV Share is 3.94% (180/4,568). This is the highest going back to where my data begins (2000).
Lamar Jackson was drafted in 2018. He has a career weighted AV of 47 so far. The sum of all players from his draft has a career AV of 2,094. His AV Share is 2.24% (47/2,094).
For the purpose of analyzing the draft. This allows me to use AV Share for active players, retired players, and even players with one year in the league. It wouldn’t be a good analysis if I use Jackson’s AV of 47 compared to Brady’s 180. However, comparing Brady’s 3.94% to Jackson’s 2.24% is a reasonable comparison.
Are there flaws in this method? Sure. This is a work in progress that I’m still thinking through. For example, Players drafted long ago will have the benefit over recent players as players from their draft are out of the league, slowing down or almost stopping the growth of the aggregate draft AV used in the denominator of AV Share. But for recent players like Mahomes or Lamar Jackson who are from a draft where there hasn’t been enough time for many existing players to start dropping out of the league, the aggregate draft AV still has nearly all of the player who made the NFL still playing and adding to the total AV used in the denominator.
While AV Share helps to compare players, that metric alone, still doesn’t help me compare players relative to the spot they were selected in. That’s where the next part of this analysis comes into play.
The next step is to calculate each draft spot’s average AV Share over the last 20 years. This gives us our second metric Expected AV Share.
To increase the sample size from 20 (I have 20 years of data) I’m using a pick range, for example, picks 1–5, 5–10, 10–20, and so on.
Here is the Expected AV Share of each pick range:
This information by itself is somewhat interesting This alone doesn’t tell me a ton, but it does suggest the draft is not a “crapshoot” like many like to say. There is a steady and consistent decline in AV Share the later the picks are. Common sense would suggest this, but it’s interesting that not one single pick range deviates from the downward trend. Where this gets useful is to compare actual picks to the spot they were selected in. This can help determine which teams draft the best, which positions are more/less predictive than others, or if some teams/GMs do better in early, mid, or late rounds.
The next part of the player evaluation is to compare the player’s AV Share to the Expected AV Share for the pick range they were selected in. This isn’t used to compare the players, AV Share can do that, this is used to compare the actual selection. Continuing with the examples I started with:
Tom Brady has an AV Share of 3.94%. He was selected 199th overall. This pick spot has an average AV Share of 0.18%. So, this pick by the Patriots had a difference of +3.76% points (3.94%-0.18%).
Lamar Jackson has an AV Share of 2.24%. He was selected 32nd overall. This pick spot has an average AV Share of 0.79%. This pick by the Ravens had a difference of +1.45% points (2.24%-0.79%)
Remember, the +/- difference is not a reflection of the player, it’s a numerical evaluation of the pick. A player can theoretically be a good player but have a negative AV Share gap if they were selected higher relative to their performance. For example, Marcell Dareus has been a solid player with 7 seasons as a starter, 2 Pro Bowls, 1 AP All-Pro, but he has an AV Share gap of -0.06% points because he was selected 3rd overall (his AV Share of 1.23% minus the Avg AV for his pick spot 1.29%).
Now that I described the process and player/pick calculations. The real goal of this analysis is to determine which teams drafted the best over the last 20 years. Using something like total AV, is flawed because it doesn’t adjust for the volume of picks or where those selections were made. Good teams like the Patriots and Steelers are rarely picking in the top 10 so pick range needs to be a part of the analysis. In this analysis, I’m taking the average +/- gap for all picks across all years for each team. Since I’m using the average, the volume of picks isn’t a factor and since this measures the gap between player and spot, that normalizes every pick regardless of if it’s top 10 or 250th.
Here is my ranking of draft success by team from 2000–2020 based on average AV Share difference per pick (ranked from best to worst):
The teams in the top five were helped the most by the following draft picks (Top 3 in AV Share Gap):
- BAL: Yanda at pick number 86, E.Reed at 24, and L.Jackson at 32
- GB: A.Rodgers at pick number 24, Bakhtiari at 109, and J.Sitton at 135
- PIT: Roethlisberger at pick number 11, A.Brown at 195, B.Keisel at 242
- CAR: Peppers at pick number 2, C.Newton at 1, and S.Smith at 74
- NO: J.Evans at pick number 108, Kamara at 67, and R.Ramczyk at 32
The bottom five teams were negatively impacted the most by the following draft picks (Bottom 3 in AV Share Gap)
- CLE: J.Gilbert at pick number 8, T.Richardson at 3, C.Coleman at 15
- OAK/LV: J.Russell at pick number 1, D.Gibson at 28, D.J.Hayden at 28
- DET: C.Rogers at pick number 2, J.Okudah at 3, M.Williams at 10
- LAR: J.Smith at pick number 2, T.Hill at 15, J.Kennedy at 12
- TB: G.Adams at pick number 4, C.Williams at 5, B.Price at 35
This next section shows the best and worst picks from 2000–2020 based on the players AV Share minus the Avg AV for their selection range.
15 best draft picks since 2000
15 worst draft picks since 2000
Using this same data (AV share vs eAV Share), I evaluated the exact median player at each draft spot in the first round. Who is the player at the center of every spot? Here is the list using data from 2000–2020 (N=21 for each spot)
Each team has many hits and misses, and most picks fall somewhere in between. For every Dak Prescott there is a JaMarcus Russell. Here are each team's best and worst picks from 2006–2020.
This is my first attempt at creating this AV Share / Expected AV Share metric and using it to evaluate the draft. In future posts, I plan to expand on this by uncovering the teams that excel (or fail) at drafting certain positions, or stages of the draft (early/late rounds), team time-series by year (and GM).
I will also explore which positions are more or less consistent relative to their draft spot. In other words which positions are more predictable in terms of draft evaluation vs career performance. Hopefully, you have found the process and findings interesting. I’m sure it will be tweaked and improved along the way.
Notes and Sources:
Source: Pro-Football-Reference
Pro Football Reference has been working through some recent bugs on their AV metric (late June 2021). As they made adjustments some players AV moved slightly. My analysis was completed before these adjustments, so some players may have a different AV than what I used in the analysis. This shouldn’t have an impact directionally, but some ratings would change slightly.
AV is a metric created by PFR founder Doug Drinen, the Approximate Value (AV) method is an attempt to put a single number on the seasonal value of a player at any position from any year (since 1950). For my calculations, I’m using Approximate Value (Weighted). Any reference to AV is weighted, You can find out more about AV here.
Weighted AV: This is Doug’s way of balancing peak production against raw career totals; for each player, he computes the following weighted sum of seasonal AV scores: 100% of the player’s best season, plus 95% of his 2nd-best season, plus 90% of his 3rd-best season, plus 85% of his 4th-best season, ….And so on. (Note: “Weighted Career AV” should not be confused with “career AV”, which is just the unweighted sum of a player’s AV scores.)
