Projection Accuracy Methodology
How I measure which fantasy baseball projection systems are most accurate, and why this approach produces meaningful results for fantasy baseballers like you.
Overview
Long ago, I decided to start evaluating the accuracy of the most popular fantasy baseball projection systems that get published each year to answer the question every fantasy manager wants to know: whose projections should I actually trust on draft day?
As I dove into this, it became obvious that measuring projection accuracy is not as simple as just seeing whether a system actually predicted a player hit 27 HRs in a season or not. The reason that does not matter as much as it seems is because each projection system operates within its own statistical universe, with its own baselines for what an average player looks like. On draft day, you are simply operating within one projection's universe to try to know how each player differs from the average in that universe. So, comparing raw numbers across systems or against actual results is an apples-to-oranges comparison that can produce misleading analysis rankings.
My methodology uses Mean Absolute Error (MAE), which I'll explain further within here. The core idea is that a great projection system doesn't need to nail the exact home run total because it just needs to correctly identify which players are stars and which are bums in each stat relative to its own projections.
This analysis is specifically done with 5x5 roto stats in mind. Historical results of my analysis are available going back to 2016 on the Historical Projection Analysis page.