An analysis of the correlation between draft grades and actual production

Jason Pauley

With the draft ending, the NFL community leaves mock draft season behind and barrels into draft grade season. And we ALL love draft grades. Even the people who hate draft grades love draft grades because it gives them the opportunity to tell us how much they hate draft grades. I like them because they condense 15 hours of transactions for 32 teams and 250+ players into one easy metric for each team. At best, the grades are fairly accurate and truly indicate how well teams performed and how those draft picks will contribute in the near future. At worst, grades give us something to discuss and debate for a few weeks. I’ve been curious about the “at best” part. Will the results years later, reflect the grades given for those teams who drafted the players? In other words, do results correlate with draft grades?

I decided to look at the 2020 draft and the aggregate GPA for each team from 18 football analysts. The aggregate draft grades I used are the ones tracked by Football Outsiders. They have a great post every year covering all of the analysts' grades and creating a summary. You can find that information here.

My back-of-the-napkin analysis compares the draft grades with the performance of players drafted in 2020. You should take this analysis/chart with a grain of salt. This is rudimentary and my performance scores are arbitrary. The process I’m using is to plot grades against outcomes to see if they correlate and how strong the correlation is. Also, to look for outliers and explore what drove those outliers. How am I measuring outcome? As I said, it’s super arbitrary. But it’s quick, which is what I have time for right now. And I hope that it directionally correlated with more advanced methods.

The team performance score:

• Each player who becomes a team’s primary starter for a season gets a point. One point regardless of if they were a starter for one season or both seasons
• A player who accumulated 10 AV over two years gets a point. About 50% of players who earn the point for starting will earn this as well. This also allows for a very small group of players who aren’t identified as the teams' starter, but performed well enough to be one, to get a point (D’Andre Swift for example)
• 1 point for a player making the pro bowl (1 point regardless of how many pro bowls)
• 1 point for 1st team All-Pro (1 point regardless of how many All-Pros)

The max a player can get is 4. I then aggregate all the players drafted for each team and sum their scores. Then I simply plot that against their grades.

That’s it. Easy peasy. But, there are some flaws I need to call out. Draft capital is not considered in this analysis. It’s easier to draft a 3-point player for a team drafting in the top 5 compared to a team drafting at 32. Draft capital is an essential element that is not a part of this analysis. I have a solution, but it’s going to take a lot of time to update my model I created in the past. Click here for that methodology. I plan to redo this analysis, using my more rigorous methodology, but I’m concerned that by the time I get around to that, draft grade season is over. Other considerations or flaws are the obvious arbitrariness of my simple production score, the number of players teams draft will vary, and positional value isn’t considered here. All caveats aside. Let’s look at the accuracy of the draft graders two years later and check out some outliers as well.

Draft Grades vs Outcomes

The chart plots the average GPA across the X-Axis (horizontal) from the worst grade (Packers) to the best grade (Cowboys) in the 2020 draft. I then plot their production score along the Y-Axis (vertical). The trend line moving from the bottom left to the top right, suggests that eventual production correlates with draft grades, however, ths correlation is weak. A tighter fit of the dots huddled around the trend line would give us more confidence in the correlation, but these data points are widely dispersed. This view allows us to easily pick out the outliers like Tenessee or Miami. Outliers above the dotted line could be considered to have outperformed their draft grades. Outliers below the dotted line appear to have underperformed their grade. Teams like Greed Bay, Carolina and Cincinnati have, so far, produced results from their draft that were projected based on their GPA.

Do draft grades correlate with production? If you squint, you can see some correlation between grades and outcome. Not a strong correlation but we do see that generally, there is an average uptick in performance for teams that were graded well, and lower performance when they were graded poorly. When I get around to using my other methodology that takes draft pick spots into consideration, I hope to see a stronger correlation…or maybe it ends up weaker, I don’t know. The correlation is weak here, but there’s something. Adding more years to the study add some much-needed sample size may provide a better conclusion.

Outliers

Below are a few teams who had the best or worst draft production scores relative to their draft grades. When I take a look at individual teams and who they selected, where they selected, and how many players they drafted, it confirms that a 2nd phase of this analysis incorporating draft capital will be a more insightful analysis. For example, the Vikings drafted 15 players, they threw a lot of darts so inevitably enough darts should land to accumulate a decent amount of points. However, some of these scores reflect the team’s draft accurately. Dallas did genuinely good; the Titans did truly bad…at least through two years.

Dallas

• GPA 3.85, rank 1
• Production Score 9, ranked 2
• Score per player 1.29

Dallas drafted a Pro Bowler in the first round, in CeeDee Lamb (WR), and a Pro Bowler and All-Pro in the 2nd round when they drafted Trevon Diggs (CB). Both of these players averaged 9.5 AV. Trevon Diggs led all CBs in the NFL with an AV of 15 this season on his way to 11 interceptions. The Cowboys drafted two other eventual starters in DT Neville Gallimore and C Tyler Biadasz. Four of their seven picks ended up as regular starters.

Minnesota

• GPA 3.77, rank 2
• Production Score 9, ranked 2
• Score per player 0.60

The Vikings had a TON of picks in this draft. They ended up drafting 15 players, with two 1st round picks and three 4th round picks. All of their production came in the first four rounds. They got six starters from their first seven picks including two-time pro bowler Justin Jefferson in the first round who was 4th in the NFL in receptions and 2nd in receiving yards in 2021. Jefferson was behind only Cooper Kupp, Davante Adams, and Deebo Samuel in AV for Wide Receivers this year. Other starters from the draft were Ezra Cleveland (T), their other 1st round pick Jeff Gladney (CB), Cameron Dantzler (CB), DJ Wonnum (DE), and 132 overall pick Troy Dye who made significant starts in his rookie year at LB.

Miami

• GPA 3.26, rank 11
• Production Score 10, ranked 1
• Score per player 0.90

Miami had three first round picks, and the draft graders took that into consideration, I’m sure. They drafted talented players, but their draft capital suggests they should have gotten a lot of talent…and they did. Their score is one that I think would be different if draft capital were incorporated into this analysis. They got Tua, so they get twheproduction points for AV and being a starter, I guess. They drafted another starter in the first round in Austin Jackson (T). Four more starters came from rounds 2–4 giving the Dolphins a total of six starters in this draft and four players with an AV over 10 across both seasons. 6’7’’, 330 lb. monster Raekwon Davis (DT) made the all-rookie team and started both seasons. Robert Hunt (G), Brandon Jones (S) and Solomon Kindley (G) all became starters out of this draft as well.

Some teams that underperformed their grades were Arizona who received the third highest grade in the draft, Buffalo, and Tennessee who received draft ratings towards the middle of the pack, but way underperformed based on performance scores.

Arizona

• GPA 3.72, rank 3
• Production Score 3, ranked 22
• Score per player 0.50

The analysts really liked the Cardinals draft, ranking them 3rd overall as a result of grabbing consensus top 5 player Isaiah Simmons at 8th overall, and filling a need on OL, by drafting Josh Jones (T) in the 3rd round, who many thought graded higher than his spot. Both of these players became eventual starters for the team, but the production ends there. No other starters have come out of their 2020 draft. They have zero Pro Bowlers, and only Isaiah Simmons has averaged more than 5 AV per season.

Buffalo

• GPA 3.07, rank 14
• Production Score 2, ranked 28
• Score per player 0.28

The Bills two performance points came from the same player, 4th round pick (128th) Gabriel Davis. 16 WRs were drafted ahead of Davis, and only one (Justin Jefferson) has more receiving TDs than Davis’ 13 TDs from 2020–21. But the production ends at Davis, as zero points were accumulated across the rest of the players drafted, including their top two picks in the draft, 2nd rounder A.J. Epenesa (DE) and 3rd rounder Zack Moss (RB).

Tennessee

• GPA 2.91, rank 19
• Production Score 1, ranked 32
• Score per player 0.17

The analysts thought the Titans did okay, enough for a slightly below average 19th ranked GPA. But their gap between production and GPA is large and real. They have accumulated only 12 AV Combined across all 6 players in their draft. Their single point comes from 2nd round pick, Kristian Fulton (CB) for being a starter for one season. They have 26 combined starts across the 6 players (4.3 per player) over the two years. But the biggest story in this draft is arguably the biggest draft bust in Titans history, Isaiah Wilson (T). Wilson was drafted in the first round (28th overall) and is now out of the league. I’d like to say he was dreadful off the field and on the field, but he literally wasn’t on the field to even make that claim. His career consisted of 4 total snaps. However, he was active off the field as he found himself on the wrong side of the law a few times. He has a new career now as a rapper and we wish him well. GGBowzer is his stage name, and surely his music career will go better than his football career.

The teams mentioned above were outliers in terms of production vs GPA. The Packers are an outlier in a different way. Their production was poor, but it matched the very low expectations according to the analysts.

Green Bay

• GPA 1.22, rank 32
• Production Score 2, ranked 28
• Score per player 0.22

According to Pro Football outsiders, the Packers did nothing to address their biggest needs at positions where the talent level was elite in 2020. They traded up in the draft to pick Jordan Love, leaving many quality players on the board who could fill critical gaps on their roster. A couple of bright spots: 2nd round pick AJ Dillon (RB) had 1,000 yards from scrimmage this year, and 6th round pick, Jon Runyan (G) started on the OL this season playing in 94% of the offensive snaps; a nice quality contribution coming from a player picked 192nd in the draft. But Jordan Love in the 1st, is what killed their grade and what help their production align so well with their poor grade. 3rd round TE, Josiah Deguara hasn’t helped much, and the rest of the draft class has yet to provide anything too significant.

What I’m trying to do here by looking back at the 2020 draft is to determine if the draft graders end up being right. I’m trying to determine how much weight to put into the experts’ opinions when draft grade season is upon us. Does this answer the question that draft graders are doing a good job, that they are accurate, and grades are indicative of future outcomes? Well, probably not by looking only at only one year, and maybe not by using this particular methodology. But I believe there are a few threads of truth, we can extract from this analysis. Sometimes analysis leads to answers, sometimes it leads to more questions, and that’s okay. This might be about 20% of the former and 80% of the latter. More rigor and more sample size applied to this same foundational thought process will probably get us closer to an answer and I plan on doing more around this topic.

Notes/Sources:

Draft GPAs are from Football Outsiders

Team selections in 2020 are from Stathead

My draft evaluation analysis which incorporates draft capital (or pick spot) can be found here. It’s only updated with player stats through 2020, which is why I didn’t use it for this study.