Thanks, Tom. That’s a nice article by Mike (and fine comments by you, of course). I’ll put a link to it on the home page.
Posted by studes on 12/09 at 10:16 AM
Win Shares Replacement Level
December 02, 2004
Using Win Shares to establish a replacement level for baseball players, with some surprising results.
Last year, we spent some time developing a baseline Win Shares level for each player, and we implemented the methodology at the Hardball Times during the year. This baseline is equal to the number of Win Shares an average player would achieve, given that specific player’s playing time. This then led to “Win Shares Above Average,” an important way to interpret Win Share totals.
At the time, I felt this was an important step toward establishing replacement levels for individual players, and I sort of made a promise to myself that I would tackle the issue when I felt ready. See, replacement level is a very complex issue with no right or wrong solution; Bill James admitted as much in his Win Shares book. I believe his next version of Win Shares will include the concept of “Loss Shares,” which is different from Replacement Level, though it also provides important context to Win Share totals.
Now, I’m not claiming that I all of a sudden have great insight into this replacement level thing. But I do now have two years’ data to play with—enough to take a meaningful stab. So I’ve decided to go for it.
My general approach is the "Readily Available Talent" one taken by Keith Woolner in Baseball Prospectus 2002. Keith divided major league players into regulars and backups, and measured the distance between the two to determine replacement level. I won't go into the pluses and minuses of this approach -- Patriot's essay, cited above, does that extremely well.
But this approach is relatively easy to do (crucial word, relatively) with the Win Shares stats we've collected over the last two years. So here's what I specifically did:
- I separated all players into their primary position played (example, Mike Piazza at catcher in 2003, first base in 2004).
- I picked the top sixty players at each position (one position times thirty teams times two years) in playing time, as measured by expected Win Shares. Expected Win Shares is essentially a measure of total plate appearances, innings in the field and/or innings pitched for each player. I called these the "Regulars."
- Win Shares groups all outfielders together, so I took the top 180 outfielders (Three outfielders times thirty teams times two years). I'll talk about pitchers later.
- I then selected the same number of players, still in descending order of playing time, after the first sixty. These are the "Replacements."
- Said differently, I took the 120 players who played the most at each position and separated them into two groups based on playing time: the Regulars and the Replacements.
- Finally, I computed the Win Shares Percentage (WSP) of each group. WSP is a Win Shares rate stat, and it's simply Win Shares divided by two times expected Win Shares. You can think of it as a player's winning percentage, though it has its limitations in that sense.
- I divided the Replacements' WSP by the Regulars' WSP to determine the Replacement Level.
What did I find? Well, here's a table of the Win Shares (WS), Expected Win Shares (ExpWS), Win Shares Percentage (WSP) of each group, and the Replacement Level of each postition:
| Regulars | Replacements | Replacement Level |
|||||
|---|---|---|---|---|---|---|---|
| Position | WS | ExpWS | WSP | WS | ExpWS | WSP | |
| Outfield | 3058 | 2523 | 0.606 | 724 | 858 | 0.422 | 70% |
| Second Base | 906 | 818 | 0.554 | 243 | 319 | 0.380 | 69% |
| First Base | 1067 | 865 | 0.617 | 246 | 296 | 0.416 | 67% |
| Catcher | 748 | 698 | 0.536 | 200 | 286 | 0.350 | 65% |
| Shortstop | 920 | 871 | 0.529 | 143 | 220 | 0.325 | 61% |
| Third Base | 915 | 841 | 0.544 | 191 | 287 | 0.334 | 61% |
A little more background: When I started making up Replacement Levels last year, I started at 50%, just on gut feel. Then, I later changed my mind and swithced to 75%. Gut feel, again. Some gut. The answer was in between. Specifically, it looks like the Replacement Levels for outfielders and second basemen is around 70%, shortstops and third basemen around 60%, and catchers and first basemen in between.
This actually might explain a number of things. Why second basemen seem to be so underpaid, for instance (there are more backups available). Or why some shortstops have received nice contracts this offseason (not enough alternatives).
Having said that, I know there are all sorts of problems with this analysis. Injuries, poor distribution of talent and questionable playing time decisions all affect these calculations. Also, two-year samples are not really definitive. And this type of analysis is very sensitive to the number of players you select for each group.
But my gut (!) tells me that the right replacement level for Win Shares is between 60% and 70%, and I'd use 65% for all position players, keeping some of these specific ranges in mind.
What about pitchers, you ask? Good question. I took a slightly different approach with them, and I only used one year's worth of data. And, mimicking Woolner again, I separated pitchers into starters and relievers. The specific steps:
- First, I selected all pitchers who started a game in 2004, and rank ordered them by Games Started. I selected the top 150 (five starters times 30 clubs) and made them the "Regulars."
- For the Replacement group, I included all pitchers who made at least two starts, or one start if that was the only game they pitched. This resulted in 140 Replacements -- slightly less than the Regulars group.
- For relievers, I rank ordered them by (Games Relieved plus Save-Equivalent Innings). I did this because expected Win Shares includes the impact of saves and holds, and pitchers on winning teams have more opportunities for saves and holds. On the other hand, I wanted to make sure pitchers who pitched key innings were ranked highly. Combining two figures like this led to a fairly representative sample across all teams.
- I then selected the top 120 pitchers (four pitchers in the bullpen for 30 teams) and labeled them Regulars and next 120 were the Replacements.
- If a pitcher started more than ten games, or started more games than relieved, I excluded him from the Replacements group.
- For both groups, I then did the WSP analysis to establish a Replacement Level for each.
Here are the surprising results:
| First Group | Replacements | Replacement Level |
|||||
|---|---|---|---|---|---|---|---|
| Position | WS | ExpWS | WSP | WS | ExpWS | WSP | |
| Starters | 1444 | 1362 | 0.530 | 126 | 309 | 0.204 | 38% |
| Relievers | 415 | 325 | 0.704 | 236 | 279 | 0.440 | 62% |
The replacement level for relievers is within the same range as position players (62%), but starting pitchers present an entirely different story (38%)! In a sentence, good starting pitchers are hard to find. I should reiterate that there are issues with this approach; all of the previous caveats apply, plus I only used data from one year. My next step will be to pull 2003 data and run the same analysis.
But if this analysis holds water, I will add replacement levels to the Win Shares tables, using something conservative like 45% for starting pitchers and 65% for everyone else. We'll call this Win Shares Above Replacement (WSAR), and I like to think it will be a major Win Share step forward.
One of the major Win Share complaints is that they underrate good starting pitchers, and this analysis seems to bear that out. But I might characterize the situation differently.
Win Shares are an attempt -- and a pretty decent one -- to assign each player's contribution to his team's wins. It doesn't matter what position the player plays, or whether he contributes with his arm, glove or bat. It doesn't matter how hard it is to do what he did, or how rare his particular skill is. What matters, within the parameters of each game and how it was played, is what he contributed to the win.
But if a player can do something that contributes to a win, and very few other people can do it, doesn't that make him more valuable? Said differently, what if Player One contributes 20 Win Shares, and there are a bunch of guys who could only contribute 10 in his place; isn't he worth more than the player who contributes 20 Win Shares, but is backed up by a bunch of guys who could contribute 15 in his place?
Well, yes, to answer a rhetorical question. Yes he is. And this is what WSAR is meant to measure. Win Shares measures how much a player contributed to his team. WSAR measures how rare his talent is among players who play the same role. Contracts are driven by WSAR, because contracts are subject to the laws of supply and demand just like any other market. What's rare is valuable, what's common is less valuable.
Both stats are useful; they measure different things.
If WS undervalues great fielders (which I accept), and undervalues good starting pitching (which I think most would agree), where will we find enough WS to fix this? Reducing the hitter % from 52% to 50% clearly isn’t enough.
The problem (as with other aspects of WS) lies in James’ insistence on making WS a bottom-up system with only positive values. That’s not a big problem for hitting, and perhaps pitching, but it’s a disaster for fielding. Clearly, many players—perhaps half—field at a below-replacement level. But no one gets fewer than zero WS in any category. Unless you give out negative fielding WS (a lot of them), you can only give Ozzie his due by stealing WS from hitters or pitchers, and then something else in the WS gets out of whack.
Posted by Guy on 12/10 at 06:58 AM
Of course Win Shares has tens of problems. The range of fielding runs, from top to bottom, should be plus/minus 20 to 30 runs, or a range of about 50 runs, for any given player. Or, 17 Win Shares difference. WS comes nowhere close to this.
If James were to come and accept negative Win Shares, half his problems go away.
But, James, as a matter of principle, will never accept a negative win share.
I once asked him about negative loss shares, and he said that if I ended up with a negative loss share, then I did something wrong.
Posted by tangotiger on 12/11 at 10:09 PM
| <<Previous Article: Proving Moneyball | Next Article: Shortstops and Defensive Win Shares>> |