#1: Who Will Make the Playoff?
First a semi-tangent to build context. After the field was announced last year, I took to the internet with great haste to diagnose precisely what happened. That effort produced this paragraph, which I like very much:
"In summation, the selection committee appears to be a little better than the polls, which still have FSU above Oregon. I will happily take whatever improvements I can. The committee also appears to be a touch better than its basketball equivalent, as there at least is no dreadful RPI to worry about. There are legitimate concerns about the metrics the committee is using, the lack of diversity on the committee (in terms of age and experience - almost everyone is a football coaching and/or administrating lifer), and unconscious biases towards the blue bloods of the sport. That said, we at least have a framework that allows for a more reasonable number of teams to compete for a title, and a methodology for arriving there that most people are at least somewhat happy with. All in all, I think the first year of the playoff went pretty well, and I'm excited for the future."
There was much justified gnashing of teeth, but regardless, the playoff and the committee are a step in the right direction. Time will tell just how good of a step it is, but through one year we have a decent idea of what the committee is looking for in a playoff participant. As best as I can tell, the committee cares about these things, in this order:
1. Very few (or no) losses
2. Strength of schedule
3. Wins over top 25 teams
4. Conference titles
Given that we have one year of data on the committee's actual thought process, we don't yet have enough to build any sort of elaborate models based on this for the sake of predicting 2015's playoff participants. That said, I think we can simplify that list such that we actually do have enough info.
Let's start at the bottom. Conference titles coincide with each of the three items above them. A conference title (at least in the Autonomy 5 or whatever the hell we're saying now) will generally give you a win over a top 25 team. Beating said team will probably boost your strength of schedule. And finally, by simply avoiding a loss in the title game, you will have very few (or no) losses. Future selection committee results may force us to re-visit this (perhaps an 11-2 Oregon will beat out an 11-1 TCU that finishes behind Baylor), but for now I think the phenomena of conference championships gets well enough wrapped up in everything else above it on the list.
Next up is wins over top 25 teams. As I said in the previous post, I think this is the main reason that Florida State leapt over TCU in the final poll. Through this, and through what the committee directly said, it became evident that this is a driving force in their evaluations. Once again though, I think the majority of this is wrapped up in the items above it on the list. A win over a top 25 team will certainly boost your schedule strength, and of course, it will help you to avoid losses. In the end, Florida State had fewer losses and a similar SOS to TCU that could explain the difference without looking at "big wins." Once again, if we see something happen this year (let's say a 10-2 non-champ Alabama beating out a 11-1 Michigan State because of Alabama's crazy stupid schedule), then I can explore adding something that measures this. But for now, I shall disregard it.
Next up is strength of schedule. Unlike the last two paragraphs, I don't think this is something we should eliminate from our model. Instead, I want to call it what it is: Power 5 conference membership. It seems clear to me that the committee isn't agonizing terribly over the differences in SOS, and rather just needs teams to clear certain barriers to be included.* Ohio State jumped over TCU and Baylor in spite of a pretty ugly schedule. From this, it seems clear that even though it was ugly, it was enough. Furthermore, Marshall was about as good of a mid-major team as we've seen in a while, but the committee took their sweet time ranking them. If 2014 Marshall couldn't at least get a little love, it seems clear that nothing short of a perfect storm will allow a mid-major access to the playoff.
*Remember the "scheduling intent" comment from Barry Alvarez? That's kind of what I'm getting at here. Be in a big conference and schedule something in the non-conference and you'll be fine.
Finally, we have number of losses. It's pretty clear that the committee cares about this at the expense of almost everything else. Sure, they were wise enough to put Oregon and Alabama ahead of a once-in-a-generation weirdo like 2014 Florida State, but they still put the Noles ahead of superior teams like Ohio State and TCU. Also, they had a two-loss Michigan State ahead of a three-loss Ole Miss, who was the best team in the nation for two months. It's pretty clear that it takes rather extraordinary circumstances for the committee not to make its initial sort simply the number of losses a team has. 2015 gives us a great team with the hardest schedule in the country (Alabama), so there may be an opportunity for the committee to elevate a deserving (used non-pejoratively) 10-2 or 9-3 Tide team above other teams. That said, I'll believe it when I see it.
Now that we've gone through that thought exercise, we are left with the following:
1. Number of losses
2. Major conference membership
This is significantly easier to model. The problem is that we still only have one year of the committee's decisions to deal with. If only we had a very similar system in place for the 16 previous years that ultimately focused on the same two things. Oh wait, we did. Thankfully, that thing (the BCS if you are weirdly unfamiliar) gives us a far better sample of data from which to pull.
The first thing I did was pull all seasonal data from 1998 to 2014, and sync up the results with the teams that finished in the top 4 of whatever system was in place at the time. This gives us 68 playoff participants, of which we can glean a little bit of information. Namely, I can break these teams down into four groups:
Major conference, no losses: 22 teams (of 23 possible - 95.7%)*
Major conference, one loss: 35 teams (of 62 possible - 56.5%)**
Major conference, two losses: 9 teams (of 111 possible - 8.1%)**
Non-major conference, no losses: 2 teams (of 11 possible - 18.2%)
*The one that didn't make the top four is 2012 Ohio State, who was on probation. I considered removing them from the sample set, but it could happen to another team at pretty much any point, so I decided to leave it in. I mean, Florida State was closer to missing the playoff than being #1 last year, so you don't even need to be on probation to have to sweat it out.
**Fun trivia: It's a dumb college football bromide that early losses don't matter. It's dumb in part because these teams that achieved great success went 67-1 in their first game of the season. If you can guess the one team that lost their opener, I will buy you a beer. Caveat: You must attend the ND-Georgia Tech game to collect your prize.
From those numbers, it's pretty clear that simply looking at number of losses and conference affiliation will paint a reasonably accurate picture of who is going to make the playoff. I may add wrinkles in future years, should a clear pattern arise in the committee's methodology. For now I feel that this is a sufficient goal to model for the purposes of projecting playoff participants.
Now that I've discussed the inputs and the theory, we have to deal with the "how." That is, how do I use this information to output the playoff probability for each team? The idea behind doing a Monte Carlo simulation* of many seasons to find an overall average outcome is pretty straightforward; I just have to determine which inputs I want to use. I'll start with the data shown above, and use that as the final step in my projections. If a major conference team goes 10-2, they'll have an 8.1% chance of making the playoff in my model. Likewise for the other three groups and their probabilities (all other teams will have a 0% chance). In order to predict the likelihood of each team falling in those four buckets, I ran 10000 simulations of each team's season, and then treated those results as the likelihood of each team falling into each bucket. For example, if TCU went 10-2 2000 times, I gave them a 20% chance of finishing 10-2 (and thus a ~2% chance of making the playoff under such circumstances).
*I went the simulation route because it's way easier than figuring out the odds of finishing 10-2 mathematically.
To do this Monte Carlo simulation, I needed two things: A rating system expressed in points above/below average (similar to SRS), and a model to predict the likelihood of winning based on the point spread.* For the rating system, I used ESPN's FPI for a few reasons. One, I like a lot of what they've been doing lately in the football sphere, whether it's hiring Brian Burke (aka. the Advanced Football Analytics guy), or revising QBR to cater less to the "clutchy" crowd. Two, it's expressed in points above/below average, which is easy to interpret. Three, they already have a pre-season set of ratings out, unlike Sagarin and some others. I may revisit this in the weeks and years to come, but for now I will be using their rating system to determine the true talent of a team.
*I also need a home field advantage, which I will get weekly from Sagarin's predictor. For this first set of ratings, I am using his final 2014 number of 2.69 points (nice). So, if a team is a 10 point favorite on a neutral field, they would be a 12.69 favorite against the same team at home.
As for the second part (a model to predict win probability based on point spread), I collected some college football spread data, and graphed it:
It seems pretty clear that this can be modeled by a logistic regression, so I did just that. This gave me the following formula:
Odds ratio = e^(0.002614+(0.119735*pts))
You can use the odds ratio to then predict the probability for an individual point as such:
Winprob(pts) = Odds ratio(pts)/(1+Odds Ratio(pts))
A standard gambling rule of thumb states that a 7 point favorite should have a 70-75% chance of winning the game. Let's test this:
Odds ratio(7) = e^(0.002614+(0.119735*7)) = e^0.84 = 2.318
Winprob(7) = Odds ratio(7)/(1+Odds Ratio(7)) = 2.318/(1+2.318) = 0.6986
Sure enough, we get the result that we would normally expect. Thus, this seems to be a pretty good model, though I am open to a better one if such a model should exist.
Armed with all of this information, I was finally able to create the sim, which I had to do with the help of R (Excel can do a lot of things, but you need some nice plug-ins to be able to do simulations easily). You can view the code on Github if you truly have nothing else to do. The main thing to note about it is that these simulations are independent of what happens to other teams. I am not simulating entire seasons at once, just individual team seasons. Simulating entire seasons would be a good goal, but the complexity of doing so would be much greater, and would take a lot more code (just what I would have to write to figure out conference tie-breakers would be way more complicated than the entirety of my current code). For now, the sheer number of simulations should help to overcome any weirdness that comes as the result of doing the simulations independently.
After running this, I had the first part done (determining the likelihood of being in the four buckets), and the last part done (figuring out the odds once you're in those buckets). Now, I just needed to tie them together, with the added wrinkle of figuring out what to do about conference championship games. While we have thousands of team seasons to build our bucket probabilities from above, we only have about 40 conference championship games in the BCS/playoff era. Basing any conclusions off of those would probably be silly. For now, I am going to take a shortcut, and project that a championship game participant from one of the four buckets has a 40% chance of adding a loss to their ledger. This accounts for the fact that a 12-0 team is more likely to make the title game than a 10-2 team, as the 10-2 team is probably more likely to lose the game if they get there.* I hope to do a little research into this soon, but for now, we're going with the simple estimate.
*This makes sense, trust me. Let's say a 10-2 team has a 70% chance of making a title game, and that they have a 43% chance of winning that game (a little less than 50% since they're probably facing a team with a better record). This would give them a ~40% chance of suffering a loss outside of their regular 12-game schedule.
Once I did all this, I was done. Well sort of. Only one more paragraph, I promise. The problem I ran into was that the sum of the probabilities for all 128 teams only added to 2.5. Since there is going to be 4 playoff teams, this should sum to 4 or at least be very close to it. I went through and checked everything, and questioned all my assumptions. Everything checked out. I changed the 40% championship game loss estimate to an unreasonably low 10%, and still only got a total of 3 teams in the playoff. What tipped me off to the ultimate issue was that the system was projecting only 0.8 undefeated teams on average, when we have averaged exactly 2 teams per year over the BCS/playoff era. This made me think of the typical problem with individual team projections, which is detailed here and here. Basically, our best modeling techniques are going to pull everything toward the mean in order to give us the best accuracy for the model as a whole. This means it will not project many if any 12-0 teams. But we still know that every year, teams ride a combination of outperforming expectations and lucky bounces to undefeated seasons. There are two ways to account for this: You can either adjust your model, or you can adjust your results. The latter is perhaps the less robust of the two methods, but it's also way easier, so I went with it. I multiplied each teams' playoff odds by 4/2.5 and got my final results:
| Rank | Team | FPI Rank | LOSS0 | LOSS1 | LOSS2 | POFF Prob |
| 1 | Ohio State | 1 | 19.90% | 37.47% | 27.96% | 68.061% |
| 2 | Baylor | 2 | 10.91% | 30.79% | 34.28% | 49.051% |
| 3 | Texas Christian | 3 | 10.14% | 28.64% | 32.19% | 45.653% |
| 4 | Georgia | 5 | 4.79% | 18.09% | 29.08% | 27.507% |
| 5 | Louisiana State | 4 | 2.57% | 12.13% | 24.72% | 18.127% |
| 6 | Oregon | 8 | 2.36% | 11.50% | 23.48% | 17.077% |
| 7 | UCLA | 12 | 2.16% | 11.39% | 23.49% | 16.671% |
| 8 | Michigan State | 15 | 1.84% | 11.55% | 25.31% | 16.566% |
| 9 | Notre Dame | 16 | 1.94% | 11.44% | 23.70% | 16.409% |
| 10 | Florida State | 19 | 1.93% | 9.79% | 22.53% | 14.743% |
| 11 | Oklahoma | 11 | 1.06% | 6.94% | 19.58% | 10.451% |
| 12 | Clemson | 20 | 1.22% | 6.97% | 17.40% | 10.441% |
| 13 | Texas A&M | 6 | 1.17% | 6.58% | 17.78% | 10.062% |
| 14 | Southern California | 10 | 0.98% | 5.94% | 16.00% | 8.958% |
| 15 | Wisconsin | 36 | 0.52% | 5.08% | 15.72% | 7.436% |
| 16 | Tennessee | 14 | 0.61% | 4.99% | 14.78% | 7.365% |
| 17 | Stanford | 17 | 0.66% | 4.61% | 13.94% | 6.996% |
| 18 | Mississippi | 9 | 0.52% | 4.47% | 13.58% | 6.603% |
| 19 | North Carolina | 31 | 0.44% | 3.08% | 10.89% | 4.884% |
| 20 | Alabama | 7 | 0.19% | 2.47% | 8.62% | 3.648% |
| 21 | Virginia Tech | 25 | 0.24% | 2.24% | 8.84% | 3.541% |
| 22 | Arkansas | 13 | 0.20% | 1.79% | 7.61% | 2.917% |
| 23 | Michigan | 30 | 0.13% | 1.53% | 6.91% | 2.476% |
| 24 | Boise State | 35 | 8.09% | 23.94% | 31.24% | 2.358% |
| 25 | Penn State | 37 | 0.10% | 1.28% | 7.52% | 2.279% |
| 26 | North Carolina State | 44 | 0.11% | 1.22% | 6.22% | 2.089% |
| 27 | Oklahoma State | 24 | 0.08% | 1.14% | 6.23% | 1.964% |
| 28 | Arizona State | 21 | 0.09% | 1.01% | 4.63% | 1.653% |
| 29 | Arizona | 29 | 0.07% | 0.86% | 5.06% | 1.544% |
| 30 | Miami (FL) | 34 | 0.06% | 0.90% | 4.59% | 1.501% |
| 31 | Nebraska | 45 | 0.06% | 0.90% | 4.50% | 1.495% |
| 32 | Missouri | 26 | 0.01% | 0.60% | 3.38% | 1.002% |
| 33 | West Virginia | 27 | 0.03% | 0.52% | 3.38% | 0.956% |
| 34 | Auburn | 18 | 0.04% | 0.51% | 3.03% | 0.917% |
| 35 | Pittsburgh | 42 | 0.05% | 0.44% | 2.49% | 0.809% |
| 36 | Georgia Tech | 28 | 0.04% | 0.43% | 2.46% | 0.773% |
| 37 | Louisville | 43 | 0.05% | 0.34% | 2.12% | 0.661% |
| 38 | Florida | 22 | 0.01% | 0.25% | 2.35% | 0.541% |
| 39 | Iowa | 54 | 0.04% | 0.26% | 1.88% | 0.533% |
| 40 | Texas | 23 | 0.00% | 0.27% | 1.71% | 0.467% |
| 41 | Cincinnati | 47 | 1.59% | 8.60% | 20.16% | 0.463% |
| 42 | Kansas State | 40 | 0.00% | 0.20% | 1.28% | 0.347% |
| 43 | Marshall | 58 | 1.16% | 6.60% | 17.38% | 0.338% |
| 44 | Appalachian State | 64 | 0.86% | 9.29% | 24.71% | 0.251% |
| 45 | California | 32 | 0.01% | 0.09% | 1.01% | 0.225% |
| 46 | Utah | 41 | 0.00% | 0.05% | 0.75% | 0.141% |
| 47 | Duke | 62 | 0.01% | 0.06% | 0.52% | 0.130% |
| 48 | Kentucky | 46 | 0.00% | 0.06% | 0.43% | 0.110% |
| 49 | Minnesota | 49 | 0.00% | 0.05% | 0.45% | 0.102% |
| 50 | Texas Tech | 38 | 0.00% | 0.04% | 0.49% | 0.100% |
| 51 | Louisiana Tech | 50 | 0.34% | 3.31% | 12.83% | 0.100% |
| 52 | Mississippi State | 33 | 0.00% | 0.03% | 0.39% | 0.078% |
| 53 | Temple | 52 | 0.20% | 2.16% | 9.60% | 0.058% |
| 54 | Northwestern | 57 | 0.00% | 0.02% | 0.26% | 0.051% |
| 55 | Illinois | 56 | 0.00% | 0.01% | 0.19% | 0.036% |
| 56 | Ohio | 85 | 0.11% | 0.69% | 3.32% | 0.033% |
| 57 | Purdue | 61 | 0.00% | 0.00% | 0.25% | 0.033% |
| 58 | South Carolina | 39 | 0.00% | 0.01% | 0.16% | 0.031% |
| 59 | Arkansas State | 65 | 0.09% | 1.57% | 9.36% | 0.026% |
| 60 | Washington | 48 | 0.00% | 0.01% | 0.08% | 0.015% |
| 61 | Western Kentucky | 59 | 0.05% | 0.61% | 4.05% | 0.014% |
| 62 | San Diego State | 76 | 0.05% | 0.68% | 4.52% | 0.014% |
| 63 | Indiana | 71 | 0.00% | 0.01% | 0.06% | 0.013% |
| 64 | Toledo | 68 | 0.04% | 0.98% | 4.81% | 0.012% |
| 65 | Memphis | 77 | 0.04% | 0.44% | 3.08% | 0.012% |
| 66 | Houston | 84 | 0.03% | 0.43% | 2.58% | 0.009% |
| 67 | Syracuse | 70 | 0.00% | 0.00% | 0.07% | 0.009% |
| 68 | Utah State | 69 | 0.02% | 0.51% | 2.90% | 0.007% |
| 69 | Maryland | 66 | 0.00% | 0.00% | 0.05% | 0.006% |
| 70 | Brigham Young | 51 | 0.02% | 0.09% | 1.08% | 0.006% |
| 71 | Boston College | 73 | 0.00% | 0.00% | 0.04% | 0.005% |
| 72 | Western Michigan | 63 | 0.02% | 0.22% | 2.20% | 0.005% |
| 73 | Washington State | 55 | 0.00% | 0.00% | 0.03% | 0.004% |
| 74 | Colorado State | 88 | 0.01% | 0.16% | 1.26% | 0.003% |
| 75 | Georgia Southern | 74 | 0.01% | 0.65% | 5.55% | 0.003% |
| 76 | Louisiana-Lafayette | 92 | 0.01% | 0.21% | 2.17% | 0.003% |
| 77 | Texas State | 94 | 0.01% | 0.17% | 1.50% | 0.003% |
| 78 | Virginia | 60 | 0.00% | 0.00% | 0.02% | 0.002% |
| 79 | Vanderbilt | 53 | 0.00% | 0.00% | 0.02% | 0.002% |
| 80 | Northern Illinois | 80 | 0.01% | 0.33% | 2.83% | 0.002% |
| 81 | Florida International | 93 | 0.01% | 0.02% | 0.29% | 0.002% |
| 82 | Ball State | 78 | 0.01% | 0.27% | 2.00% | 0.002% |
| 83 | San Jose State | 87 | 0.01% | 0.06% | 0.50% | 0.002% |
| 84 | Rutgers | 79 | 0.00% | 0.00% | 0.01% | 0.001% |
| 85 | Colorado | 67 | 0.00% | 0.00% | 0.01% | 0.001% |
| 86 | Wake Forest | 86 | 0.00% | 0.00% | 0.01% | 0.001% |
| 87 | Central Florida | 75 | 0.00% | 0.10% | 1.54% | 0.000% |
| 88 | East Carolina | 82 | 0.00% | 0.09% | 0.62% | 0.000% |
| 89 | Rice | 96 | 0.00% | 0.01% | 0.25% | 0.000% |
| 90 | Navy | 89 | 0.00% | 0.29% | 2.69% | 0.000% |
| 91 | Oregon State | 81 | 0.00% | 0.00% | 0.00% | 0.000% |
| 92 | Air Force | 104 | 0.00% | 0.00% | 0.11% | 0.000% |
| 93 | Nevada | 97 | 0.00% | 0.02% | 0.27% | 0.000% |
| 94 | Central Michigan | 101 | 0.00% | 0.00% | 0.02% | 0.000% |
| 95 | Iowa State | 72 | 0.00% | 0.00% | 0.00% | 0.000% |
| 96 | Texas-El Paso | 114 | 0.00% | 0.00% | 0.01% | 0.000% |
| 97 | Middle Tennessee State | 83 | 0.00% | 0.07% | 0.65% | 0.000% |
| 98 | Kansas | 109 | 0.00% | 0.00% | 0.00% | 0.000% |
| 99 | Fresno State | 106 | 0.00% | 0.00% | 0.04% | 0.000% |
| 100 | Hawaii | 102 | 0.00% | 0.00% | 0.02% | 0.000% |
| 101 | Akron | 115 | 0.00% | 0.00% | 0.02% | 0.000% |
| 102 | Old Dominion | 110 | 0.00% | 0.01% | 0.22% | 0.000% |
| 103 | Louisiana-Monroe | 105 | 0.00% | 0.00% | 0.02% | 0.000% |
| 104 | Bowling Green State | 91 | 0.00% | 0.01% | 0.07% | 0.000% |
| 105 | South Alabama | 122 | 0.00% | 0.00% | 0.00% | 0.000% |
| 106 | Wyoming | 116 | 0.00% | 0.00% | 0.02% | 0.000% |
| 107 | New Mexico | 107 | 0.00% | 0.02% | 0.13% | 0.000% |
| 108 | Buffalo | 119 | 0.00% | 0.00% | 0.00% | 0.000% |
| 109 | North Texas | 111 | 0.00% | 0.00% | 0.01% | 0.000% |
| 110 | Florida Atlantic | 108 | 0.00% | 0.01% | 0.05% | 0.000% |
| 111 | Tulane | 90 | 0.00% | 0.09% | 0.75% | 0.000% |
| 112 | Texas-San Antonio | 126 | 0.00% | 0.00% | 0.00% | 0.000% |
| 113 | Massachusetts | 95 | 0.00% | 0.01% | 0.19% | 0.000% |
| 114 | Southern Mississippi | 98 | 0.00% | 0.02% | 0.19% | 0.000% |
| 115 | Miami (OH) | 117 | 0.00% | 0.00% | 0.01% | 0.000% |
| 116 | South Florida | 103 | 0.00% | 0.00% | 0.05% | 0.000% |
| 117 | Army | 124 | 0.00% | 0.01% | 0.00% | 0.000% |
| 118 | Kent State | 99 | 0.00% | 0.03% | 0.20% | 0.000% |
| 119 | Tulsa | 100 | 0.00% | 0.02% | 0.13% | 0.000% |
| 120 | Idaho | 125 | 0.00% | 0.00% | 0.00% | 0.000% |
| 121 | Troy | 123 | 0.00% | 0.00% | 0.00% | 0.000% |
| 122 | Nevada-Las Vegas | 128 | 0.00% | 0.00% | 0.00% | 0.000% |
| 123 | New Mexico State | 112 | 0.00% | 0.00% | 0.02% | 0.000% |
| 124 | Connecticut | 118 | 0.00% | 0.00% | 0.00% | 0.000% |
| 125 | UNC Charlotte | 120 | 0.00% | 0.00% | 0.01% | 0.000% |
| 126 | Georgia State | 121 | 0.00% | 0.00% | 0.01% | 0.000% |
| 127 | Southern Methodist | 113 | 0.00% | 0.01% | 0.02% | 0.000% |
| 128 | Eastern Michigan | 127 | 0.00% | 0.00% | 0.00% | 0.000% |
Note: All of the teams from 87-128 have 0% probability according to this simulation, but I left them in to keep the probabilities of losing 0, 1, and 2 games, should you be curious (those probabilities, by the way, are unadjusted to fit the 4 playoff team model). As the season progresses, I will eliminate teams that have no chance of finishing in one of the four buckets. The first week alone will likely knock off about one quarter of the teams.
As you might guess, the majority of the top teams in FPI finish at the top of this ranking (glad I spent all that time to get such deep insights). That said we do see the SEC West take a major hit from having to play each other so much. Most notably, we see Alabama tumble 13 spots from their FPI ranking, as they'll likely have to go at least 8-2 against the ten top-40 opponents on their schedule* (including a title game). Coincidentally, their week 1 opponent Wisconsin is the biggest mover in the opposite direction, jumping 21 spots from their FPI rank to 15th largely on the merits of an easy schedule. I'm very interested to see how these rankings progress over the course of the season. For now, I think they are a good portal into what to expect from the upcoming season.
*What's most interesting about this is when you look at the SEC as a whole. The SEC is most likely the best conference in the land (sure, you can make an argument for the Pac-12, but let's not for now). And yet, it's only third most likely to put a team in the playoff (see the chart below). Part of this may be because the gauntlet will affect everyone and leave no teams looking great at the end of the season. Part of it may be that the lack of a title game gives an advantage to the Big 12, and Ohio State is enough of a juggernaut to override a whole conference by themselves. But I also think that the issue I mentioned earlier plays a part. We know someone is likely to come out of the SEC at 12-1 and make the playoff, but we really have no idea who that team is. So we lump them all near each other in the rankings, assume they'll all beat each other up, and go on our way. I would bet an SEC team does make the playoff, but it's interest to ponder what it means if none do.
| Conference | Exp Playoff Teams |
| Big 12 | 1.090 |
| Big 10 | 0.991 |
| SEC | 0.789 |
| Pac 12 | 0.533 |
| ACC | 0.396 |
| Independent | 0.164 |
| Mountain West | 0.024 |
| American | 0.005 |
| CUSA | 0.005 |
| Sun Belt | 0.003 |
| MAC | 0.001 |
#2: Which Weeks/Games are the MOST IMPORTANT?
In the middle of last season, I previewed the remainder of the season by ranking the strengths of the individual weeks. While I feel the results were solid, it was ultimately a subjective exercise. Since I now have a cool database with all of the individual game probabilities, combined with a projection of playoff likelihood, I figured I could once again do a week-to-week preview, but this time make it super nerdy.
To evaluate the interest of a given game (and thus the entire week in sum), two main options sprung to mind. Both options based the "interesting" metric on top teams losing, because let's be honest, big upsets are exciting. The first was slightly more subjective, as I simply multiplied each team's inverse FPI rank (#1 ranked team gets 50 points, #2 gets 49 points, and so on), by their probability of losing a game, and summing the results for a given week. This basically says there is some interest in any top 50 team losing. The "problem" with this is the same as the playoff probabilities from the last section: The SEC has so many quality teams that SEC games dominate the list, even though some of those games will be between teams with many losses. This isn't to say that a battle of 7-4 Missouri and 6-5 Arkansas wouldn't be a great game, but I feel this ignores the centrality of the playoff to the state of current college football.
As a result, I went with the second method, which weighed things by teams' playoff probability as opposed to their raw ranking. For each game, I calculated the predicted playoff teams lost* using the following formula:
(1 - Team A Win Probability)*(Team A Playoff Probability/3)
+
(1 - Team B Win Probability)*(Team B Playoff Probability/3)
*"Playoff Teams at Risk" would probably be a more accurate name for this concept, but I decided to go with the shorter name.
The first part is simple; I am simply calculating the probability of a team losing a given game. The second part is a little more hand-wavy. My formula for determining if a team makes the playoff calculates the odds of finishing the season with 0, 1, and 2 losses. Thus, the total playoff probability is the sum of those three things. Because we don't know which numbered loss a potential loss would be for a team, dividing the overall probability by three also gives us the average playoff probability surrendered by a loss. When we then multiply this surrendered playoff number by the loss probability, we get the overall odds of a given game knocking a team out of the playoff, so to speak.
Now, I'm well aware that this is not a perfect methodology. It obviously ignores everything that doesn't affect the playoff, so we're probably missing a little nuance. This part doesn't bother me much, because I understand that the vast majority of non-school affiliated college fandom is focused on the top of the sport. Still, I would understand complaints of this ilk. What bothers me more are some of the statistical nuances that are left out. For example, a late season game is much less likely to feature an undefeated team, so we probably be focusing more on the chance of making the playoff as a 1 or 2 loss team at that point. As I said in the earlier section, I'll probably dig more into the week-to-week data at my disposal as the season progresses. While I'm unsure how much that data will help the playoff prediction model, it may actually be really helpful for this one. If I could use the actual decrease in playoff probability from teams that went from 7-0 to 7-1, then I could make the model a bit more "exact." I'm not sure if exactness is what I would want in this situation, but I should probably at least try and find out.
One final note before getting to the list. You may notice that the summation of playoff teams lost adds to about 3.2. In theory, this should sum to 4 because the playoff has 4 teams it*. What's missing then is the results of championship week. This means that week would score about 0.8 and be the most exciting in terms of affecting the playoff field. This makes sense, as the vast majority of top teams should be in action, and almost every playoff spot will be on the line.
*More complicated explanation. The playoff probabilities are forced to sum to 4. Since the metric this formula creates reflects the potential loss of those preseason odds, all of those probabilities should sum to the same 4. For example, if Ohio State has a 70% chance of making the playoff, and all of their games carry the potential to reduce that 70% to an eventual 0% (not making the playoff), then those games should sum to 70% in the "playoff games lost" metric.
After all that noise, here is the countdown, presented in chronological order:
Week 1 - 0.174 playoff teams lost (Rank: 9th of 13)
1. Ohio State (80% win probability) at Virginia Tech - 0.055 playoff teams lost
2. Alabama (79%) vs Wisconsin - 0.022
3. Texas at Notre Dame (71%) - 0.017
The more rudimentary metric thinks Texas A&M-Arizona State is the most exciting game of the week, but these top three games clearly show what this playoff-oriented rating favors: top teams with a reasonable chance of losing. The Aggies and Devils may produce a fascinating battle, but they then have to march through the two toughest divisions in the sport, so the game is unlikely to directly affect the playoff field. Week 1 sees two of the best teams in the nation face their toughest test of non-conference play, while ND gets what should be an interesting fight. This won't compare to opening weekend 2016, but it's not bad. At least, it's not bad compared to...
Week 2 - 0.146 playoff teams lost (Rank: 12th)
1. Oregon at Michigan State (54%) - 0.056
2. Oklahoma at Tennessee (57%) - 0.030
3. LSU (78%) at Mississippi State - 0.013
Now that week 1 is a yearly showcase for hyped-up neutral site games, week 2 has become a graveyard for quality football. Week 2 is so devoid of depth that the #10 and #11 games by this metric involve Baylor and TCU playing FCS opponents. While this does seem like a pretty poor week, at least we can all agree that a couple of big non-conference games played on home fields is a pretty tantalizing proposition.
Week 3 - 0.143 playoff teams lost (Rank:13th)
1. Stanford at USC (65%) - 0.026
2. Ole Miss at Alabama (61%) - 0.018
3. Georgia Tech at Notre Dame (74%) - 0.016
There's actually a decent number of fun games this weekend, but not a whole lot in terms of contests that will affect the playoff race. Fun fact: By this metric, the Irish play the 3rd most interesting game in Week 1, the 4th most interesting in Week 2, and the 3rd most interesting in Week 3. Notre Dame's schedule strength lacks the upside to finish quite as high as some SEC and Pac-12 teams, but it should at least be the most intriguing early slate of any team in the country. In other words, if the Irish come out of September unbeaten, it's on.
Week 4 - 0.150 playoff teams lost (Rank: 11th)
1. TCU (82%) at Texas Tech - 0.028
2. UCLA (66%) at Arizona - 0.022
3. Arkansas (51%) vs Texas A&M - 0.022
Week 4 has probably the most interesting example of the difference between a ranking system that focuses on top-ranked teams and one that focuses on playoff likelihood. The #4 most important game features USC and Arizona State, which are ranked higher in FPI (10th and 21st) than their conference-mates in the #2 game (12th and 29th). Common sense should dictate that USC-ASU rates higher. The reason that it doesn't is actually quite simple: USC and Arizona State both have a major non-conference hurdle to clear*, while the other two teams do not. Because of this, UCLA and Arizona are both a little more likely to have gaudier records, and thus a little more likely to make the playoff (under my set of assumptions). In the end, all four of those teams should be great, so we're splitting hairs to say that one game is better. Luckily, it doesn't even matter because you can easily watch both games. Hooray.
*USC is also the only one of those four teams that draws both Oregon and Stanford from the North, so there's that, too.
Week 5 - 0.269 playoff teams lost (Rank: 5th)
1. Notre Dame at Clemson (52%) - 0.045
2. Alabama at Georgia (66%) - 0.039
3. Texas at TCU (85%) - 0.024
As the calendar moves into October, we finally reach the meat of the schedule. I remain angry that perhaps my two most anticipated games of the year will likely air opposite of each other, but whatever, I have two TVs. It may seem odd that a TCU home game in which they're likely to roll gets such high marks in this system, but FPI thinks Texas will be pretty good (23rd) so this game ends up in the top three of a strong weekend.
Week 6 - 0.209 playoff teams lost (Rank: 7th)
1. Georgia (56%) at Tennessee - 0.054
2. TCU (83%) at Kansas State - 0.027
3. Miami at Florida State (72%) - 0.018
The game most likely to decide the SEC East* is also one of the most meaningful games of the year in terms of playoff odds. The Vols and Dawgs could be every bit as good as the teams in the West, and since they don't have to play all of those teams, they could find themselves in very good position at the end of the season. The Nebraska-Wisconsin tussle ranks 4th for the weekend, and could also help to decide a much less important division.
*This comes with my mandated warning to not sleep on Missouri, because they're probably going to win the East every year until the heat death of the universe.
Week 7 - 0.283 playoff teams lost (Rank: 4th)
1. USC at Notre Dame (51%) - 0.042
2. UCLA at Stanford (52%) - 0.040
3. Penn State at Ohio State (91%) - 0.027
One of the best weekends of the year gives us two toss-up games and....an Ohio State blowout. I'm a little surprised we haven't seen more Ohio State games on this list yet, simply because of how high their playoff odds are. Then again, their playoff odds are so high in large part because of the ease of their schedule. A bunch of >95% win probability games don't offer a lot of chance for intrigue, and my system seems to reflect that pretty well.
Week 8 - 0.168 playoff teams lost (Rank: 10th)
1. Texas A&M at Ole Miss (52%) - 0.028
2. Florida State (53%) at Georgia Tech - 0.024
3. Tennessee at Alabama (62%) - 0.020
By this metric, week 8 is the lowest rated week of "conference season." But look at these games! These should be awesome! If we turn back to my more rudimentary metric that simply looks at what the top 50 teams are doing, week 8 soars well above week 1. But, Ohio State plays Rutgers, and Baylor plays Iowa State, and TCU plays nobody, so a good portion of the playoff picture is unlikely to change based on the results of these games. Still, watch these games...they're really good!
Week 9 - 0.176 playoff teams lost (Rank: 8th)
1. Oregon (60%) at Arizona State - 0.026
2. West Virginia at TCU (87%) - 0.023
3. Georgia (78%) vs Florida - 0.022
This is the annual "get yard work done" Saturday, as the top two games of the week both take place on Thursday night. College football is always better than no college football, but if you have to sacrifice a Saturday to the gods of your choosing, this is the one.
Week 10 - 0.291 playoff teams lost (Rank: 3rd)
1. Florida State at Clemson (58%) - 0.043
2. TCU (75%) at Oklahoma State - 0.042
3. LSU (51%) at Alabama - 0.036
As we enter November, we get the first of the truly great weekends. Alabama shows up on this list for the fifth time, and for all five games they have at least a 20% chance of defeat. If the unholy marriage between Nick Saban and Lane Kiffin produces another playoff team with this slate of games, then we will know that the devil is real, and that he loves college football.
Week 11 - 0.263 playoff teams lost (Rank: 6th)
1. Oklahoma at Baylor (72%) - 0.071
2. Oregon (51%) at Stanford - 0.040
3. Georgia (63%) at Auburn - 0.035
Week 11 isn't super deep with great games, but boy howdy is that a top three. Even thought Baylor and TCU are roughly even in terms of schedule and pre-season hype, we haven't seen them on this list much, largely because their schedule is much more backloaded than that of the Frogs. One way or another, the Bears will be the team of November.
Week 12 - 0.398 playoff teams lost (Rank: 2nd)
1. Michigan State at Ohio State (77%) - 0.094
2. TCU (57%) at Oklahoma - 0.085
3. Baylor (76%) at Oklahoma State - 0.045
4. USC at Oregon (59%) - 0.041
5. LSU (54%) at Ole Miss - 0.040
The week before Thanksgiving can often be a mirror image of week 2. As almost all big rivalry games get played the next week, many teams "take it easy" the week before with an FCS game, a lesser conference opponent, or simply a bye. That is not the case in 2015. In fact, the week is so amazing that I had to include more than three games. The last two have more potential playoff impact* than many week's #1 games. I would strongly recommend against it, but if you want to tune out college football until late November, you still might get the main gist of the season in the final three weeks.
*LSU-Ole Miss is fascinating, because it's very likely that both teams enter the game with multiple losses and faint to no playoff hopes. But if one or both come into the week in the top five, it might be the biggest game of the year.
Week 13 - 0.497 playoff teams lost (Rank: 1st)
1. Baylor at TCU (58%) - 0.159
2. Ohio State (82%) at Michigan - 0.047
3. UCLA at USC (60%) - 0.045
4. Texas A&M at LSU (64%) - 0.043
5. Notre Dame at Stanford (57%) - 0.041
Like week 12, week 13 gives us incredible depth. The main reason it far surpasses week 12 is the #1 game. Baylor-TCU is damn near twice as important to the playoff picture as any other game this season. The 0.159 score represents 4% of the whole freaking season. Given that we haven't played a single game yet, the primacy of one game seems a preposterous statement. That said, both teams appear to be set up for great seasons, and the Big 12, while decent, probably won't be the murderer's row that the SEC West and Pac-12 South should be. The combination of great teams and manageable schedules sets up this "new" rivalry as a de facto play-in game. I cannot wait.
Summary
If you would like to see all the data once, here is the full listing of weeks in the season:
| Week # | Playoff Teams Lost |
| 13 | 0.497 |
| 12 | 0.398 |
| 10 | 0.291 |
| 7 | 0.283 |
| 5 | 0.269 |
| 11 | 0.263 |
| 6 | 0.209 |
| 9 | 0.176 |
| 1 | 0.174 |
| 8 | 0.168 |
| 4 | 0.150 |
| 2 | 0.146 |
| 3 | 0.143 |
If you want to see a full listing of the top conference and non-conference games, here is that:
| Conference* | |||
| Home | Away | Week | Playoff Teams Lost |
| TCU | Baylor | 13 | 0.159 |
| Ohio State | Michigan State | 12 | 0.094 |
| Oklahoma | TCU | 12 | 0.085 |
| Baylor | Oklahoma | 11 | 0.071 |
| Tennessee | Georgia | 6 | 0.054 |
| Michigan | Ohio State | 13 | 0.047 |
| USC | UCLA | 13 | 0.045 |
| Oklahoma State | Baylor | 12 | 0.045 |
| LSU | Texas A&M | 13 | 0.043 |
| Clemson | Florida State | 10 | 0.043 |
| Non-Conference | |||
| Home | Away | Week | Playoff Teams Lost |
| Michigan State | Oregon | 2 | 0.056 |
| Virginia Tech | Ohio State | 1 | 0.055 |
| Clemson | Notre Dame | 5 | 0.045 |
| Notre Dame | USC | 7 | 0.042 |
| Stanford | Notre Dame | 13 | 0.041 |
| Tennessee | Oklahoma | 2 | 0.030 |
| Florida | Florida State | 13 | 0.027 |
| Georgia Tech | Georgia | 13 | 0.024 |
| Alabama | Wisconsin | 1 | 0.022 |
| Pittsburgh | Notre Dame | 10 | 0.018 |
*It seems that in the playoff era, conferences have begun to backload their schedules. Get used to Tennessee-Georgia being the best game of October.
#3: Who Will Really Make the Playoff?
Since it wouldn't be a college football preview without the author offering up his or her educated guess as to what will happen, I might as well end with this. I find that most rankings I have seen, both objective and subjective, appear to be pretty similar and pretty much correct, or at least as correct as pre-season rankings can be. After a season in which the four playoff teams were pretty much the top four at the beginning of the season, I don't see a reason to believe that things will be any different this year. Thus, let's go with the following:
Ohio State
Baylor
Georgia
Michigan State
Lord, are people going to be pissed off if the 1-4 game is a Big Ten rematch. I can't wait.
No comments:
Post a Comment