The ACC, The NFL Draft And Shaky Blogger Logic

You may have seen this study kicking around the internet the last couple of days:

2013 NFL Draft Recap: #TarHeels and #Seminoles on top of #ACC!NC does great job of converting talent to picks: tiny.cc/mheiww

— Manish Tripathi (@sportsmktprof) May 3, 2013

Per the author, an Emory professor/student (unclear at this point), his methodology was such: “We are examining how many picks were produced by each school, relative to their recruiting classes over the relevant corresponding period for the 2013 Draft.”

Sounds simple enough. After crunching the numbers, he determined FSU and UNC were the big winners:

“Winners: With 11 picks in the draft, the Seminoles did a good job of converting top talent (they averaged a top ten ranked recruiting class over the relevant period for the 2013 draft). The Tar Heels were the surprising winners of this draft. North Carolina had 5 picks in the draft, and did not average a top 20 recruiting class over the corresponding time period! Thus Carolina was very successful at converting high school talent into picks.”

This last part is where things start to break down for me and my radar goes off.

The author’s assertion here is the UNC coaching staff was able to groom an inordinate number of high schoolers into pros given their relatively lower talent pool.

But if you’re a State fan who follows recruiting closely, you are probably well aware of just how much talent UNC has been bringing in over the last decade or so, especially relative to the Wolfpack who recruit a similar footprint to the Heels.

The author asserts the Heels haven’t averaged a recruiting ranking of 20 or higher during “the relevant corresponding period” for this draft (which, after some wrangling with the author I learned was the ’08—’10 classes; he never states that at any point when laying out his methodology), but in 2009, the Heels landed one of their best recruiting classes ever—ranked 9th by Rivals.com. The classes of 2008 and 2010 weren’t stellar, but they weren’t lousy either—32nd and 29th, respectively.

Further, 2007′s class was ranked 17th and 2011′s 16th—another two very strong classes that bookend the author’s timeframe and weren’t considered in his assertion. Point being: the flow of talent into Chapel Hill certainly hasn’t been lacking; certainly not to the degree the author wishes to imply.

When reading this piece, I wanted the author to dig a little deeper to give us a clearer picture of UNC’s development prowess. Who were these five picks from the Tar Heels? Were they elite kids sprinkled amongst the ’08, ’09 and ’10 classes?

As it turns out, three of the five draftees were only three-star kids (again, per Rivals.com). The other two were four-stars (Sylvester Williams and Giovanni Bernard). So Carolina gets credit for putting three three-star kids into the league, which in turn supports the author’s thesis.

But wait…if the majority of UNC’s draftees were three-star prospects, what happened to the four-and-five-star kids? It certainly takes a lot more talent than a heap of three stars and a few four stars to get the kind of recruiting rankings the Heels have consistently gotten of late. Specifically, what about that top-10 class in ’09?

Here’s where I feel like the author really dropped the ball in his methodology. By downplaying Carolina’s average recruiting ranking from ’08-’10 as being sub-top-20 (23.33), we don’t get a clear sense of just how much talent hasn’t panned out in Chapel Hill.

(Let me stop right here for a second before the cries of “YER JUST ANOTHER HATES FAN NATION TOOLBAG”. 1) Five draft picks in one draft is a feat for most non-SEC schools; it’s even moreso for an NC school splitting a limited talent pool. They deserve a ton of credit for that; 2) My larger goal isn’t to slam UNC here—it’s to attack the author’s faulty logic; the primary reason UNC is the subject is because it’s one of only a small handful of colleges I give a crap about and honestly the only reason I bothered to click on this guy’s link in the first place.)

Where the author needed to go further, in my opinion, was to look at the total number of kids ranked as a 5-star, 4-star or 3-star prospects over—as he describes it—the “relevant corresponding timeframe” and see how many of those kids made it to the pros.

In doing that exercise, we see Carolina hasn’t quite lived up to the lofty praise the author wishes to bestow upon them as player developers.

That’s 25 four-star and five-star prospects in three classes, with an additional 34 three-star kids to boot. While an SEC school might scoff at a talent pool like that, it’s certainly nothing to sneeze at if you’re in the ACC.

Considering only five of these 59 potential draftees were drafted this year, is UNC’s conversion rate really something to write home about?

If anything, the 2013 draft proved you stood a better chance making the pros as one of UNC’s three-star crop (3/34) than as a five-star (0/1) or a four-star (2/24).

If your argument is “UNC develops players from middling recruits into stars,” you might have a point. If your argument is “UNC takes in elite talent and gets them into the pros,” your argument holds much less water.

Now, before the cries of “UNC-obsessed Hate-Fan” swell much further, let’s look at NC State’s output over this same time frame as a comparison:

That comes to 12 four-and-five-star recruits with another 38 three-stars in tow. About half as many elites as the Heels with a few more middlers to balance out the ledger. The Pack put two of their 12 four-stars in this year’s draft (Amerson, Glennon) with one of their three-stars (Wolff). Comparing conversion rates, State was able to improve about 100% over UNC’s mark when it came to elite talent (2/12 vs 2/25).

So which school comes out on top by looking at things a bit deeper? Is State more successful developing talent because of higher percentages or is UNC better because of volume (5 vs 3)?

The answer is…we don’t know. We can’t know. Why? Because the ultimate flaw of this author’s logic now comes home to roost: SAMPLE SIZE.

How on earth can you make declarations as definitively as “Carolina was very successful at converting high school talent into picks” when a pick or two here or there can make all the difference in the world? If Travis Bond isn’t drafted in the seventh-and-final round of the draft, suddenly UNC’s converting their high school talent 20% less successfully.

Looking at another ACC school, the author lumps Clemson into the “Middle Of The Pack” category—yet they put four picks into the draft. If Clemson gets a fifth or sixth pick, are they suddenly tremendous at converting talent where before they were merely average?

This author falls into a trap A LOT of folks encounter when trying to find new analytical methods of breaking down sports performance. Without a reasonably large sample size, you just can’t make broad-sweeping declarations without some serious hedging and caveats to go along with them. A study like this would be much better served to look at draft totals over a five or 10-year span, not just a single draft.

You also have to dig deep enough into your supporting data to make sure it backs up your claim (and not gloss over it when it doesn’t). Yes, over the three-year span of ’08-’10, UNC’s recruiting classes were averaging a sub-20 ranking. But is an average ranking of 23 appreciably worse than 20? Are you conveniently ignoring a top-10 class wedged in there that was stocked with talent? Are the classes balanced with lots of 4-and-3-star kids, or are they top-heavy with a handful of can’t-miss prospects and a ton of 3-and-2-stars? Until you look at the classes individually and examine their player-by-player makeup, you simply can’t make judgments on player development prowess like the author does here.

Lest you exit this piece feeling like it was a UNC slam job, again, kudos to the Heels for putting five guys in the draft in one year. I hope my Wolfpack can post similar numbers on a year-by-year basis. I also hope the author takes another look at his reasoning to determine if it’s as rock-solid as he believes it to be.