Batter's Box Interactive Magazine Batter's Box Interactive Magazine Batter's Box Interactive Magazine
The recent induction into the Hall of Fame of Dennis Eckersley and Bruce Sutter, and the rising vote totals for Goose Gossage, have me thinking about standards for greatness in a closer.

Isn't it enough that a closer is among the very best at his craft? No. The problem is role. Not all roles are created equal. Pinch-hitters might come to bat 70 times in a season, but can play a decisive role in the game. Strangely, here is no hue and cry to put Manny Mota or Smoky Burgess in the Hall of Fame. Obviously, the closer plays a decisive role in the game more frequently than a pinch-hitter, and has more influence than a pinch-hitter by far on the results of games over the course of a season. It is possible that closer effectiveness could be so important to game results that they could be considered to be among the greatest players to ever play the game, but it is not immediately obvious that it is so. How then do we measure the contribution of a closer compared with other pitchers?

A closer's contribution is potentially different from a starter's in the following ways:

  • the closer pitches somewhere between one-quarter and one-half of the number of innings that the starter does
  • relievers in general, and closers in particular, are more effective than starters due to the nature of their roles
  • the effectiveness of the closer per batter faced matters more to the game result than the effectiveness of the starter ("leverage")
  • measuring the effect of the loss of a closer requires an examination of the possibility of an increased role for other members of the bullpen ("chaining")
  • the 2nd and 4th factors make the evaluation of replacement level for a closer more difficult, and
  • closers can potentially make a greater post-season contribution than their regular season contribution even as compared with a top starter, due to the number and importance of the games and the extra days off in the schedule
Reliever vs. Starter Effectiveness

When I looked at John Smoltz last February, I pointed out that both he and Eckersley were both much more effective when working as closers than as starters. In July, Steve Treder of the Hardball Times took a thorough approach, looking at how pitchers who had performed in starting and relief roles for a minimum of 75 appearances after 1957 had fared in each. He found that they performed much better as relievers- 14% fewer homers, 15% more strikeouts, 7% more unintentional walks and 8% lower ERAs. For closers, the data was even more extreme- 18% fewer homers, 13% more strikeouts, 8% fewer walks and 16% lower ERAs. Treder's conclusion is that we ought to apply an effectiveness equalizer of 5 to 15% to rate stats including ERA when comparing relievers with starters. I agree with this conclusion especially in relation to closers (for whom the improvement was across the board), and will apply an equalizer of 10% for comparison purposes. This is a conservative estimate of the effect.

As an aside, why is it that closers improve on their performance in the "unintentional walk" department as compared with their work as starters, whereas other relievers deteriorate in this respect? I would suggest one possible answer. Non-closer relievers enter the game more often with runner(s) on and first base open, making the "pitch-around" walk a more common element of their games.


We do know that closers come into games at different stages of the game, depending on the era and depending on the manager. We have easily accessible statistics for 2002-06, courtesy of Here is the card for Mariano Rivera. The pLI is the average leverage of opponent's plate appearance and ranges from 1.82 to 2.23 over the years with an average of 1.99. In Joe Nathan's time with the Twins, his pLI has ranged between 1.62 and 1.92. Baseball Prospectus uses a somewhat different definition of leverage, and has slightly different numbers. For instance, BP has Joe Nathan's leverage index in 2004 as 2.06. The average fangraphs leverage for closers in 2006 and 2005 was 1.96 and 1.93 respectively; using the BP definition, it ws 1.86 and 1.83. For comparison, starters typically have a leverage index of about 1.

To put it together, have a look at John Smoltz' fangraphs card. While working as a closer during 2002-04, his average leverage was 1.91; when he returned to the starting role in 2005-06, his average leverage was 1.05. If we assess closers as having double the leverage of starters in general, we are being generous to them.


The loss of a starter impacts a team differently than the loss of a closer. If a starter is lost, the team must simply replace the starter. If a closer is lost, the replacement pitcher will not be placed in the closing role. This means that the loss of performance cannot simply be measured by the difference in performance between the closer and the replacement multiplied by the leverage. It is time for an example. Rather than choosing potential Hall of Famers, I thought that it might be better to stay closer to home, by taking a closer look at Tom Henke and Jimmy Key , during their primes in Toronto from 1985-1992. Besides, I like both of them. Cue Glory Days, Mr. D.J.

The 1985 club provides a good illustration of chaining. The pen that year, aside from Henke, consisted of Bill Caudill, Gary Lavelle, Jim Acker and Dennis Lamp. Caudill had struggled in the closing role, when Henke came up, but was a pretty good pitcher as were Lavelle, Acker and Lamp. Henke's leverage index, according to Baseball Prospectus in 1985 was 1.57. If Henke had been lost, Lavelle or Caudill would have reassumed the closing role, the other 3 pitchers would have moved up, and Ron Musselman or Steve Davis or a young John Cerutti would have moved into the low leverage mop-up role. The loss of Henke would have been equal to the difference between Henke and Lavelle say (large, but Lavelle was substantially above replacement level) at a leverage of 1.57, and the difference between Dennis Lamp and Ron Musselman at a lower leverage (Lamp's leverage was 1.15 that year).

So, when we assess what Henke's value, we cannot simply multiply the difference between his performance and replacement performance by his leverage. The actual difference will depend on the remaining members of the pen, but squaring things off at between 75% and 85% of leverage seems to me to be fair.

The situation is different for starters. Roy Halladay and Gustavo Chacin have essentially the same leverage when they pitch (as starters). The same was true for Jimmy Key, Jim Clancy and Luis Leal. If Key was lost in 1985, it would have meant more starts for Luis Leal, or some other replacement level pitcher.

Post-season and the wonder that is Mariano Rivera

Bob Gibson is considered by many to be the greatest post-season pitcher ever. He was superb indeed, making 9 starts and throwing 8 complete games and 81 innings, while posting a sweet 7-2 record with a 1.89 ERA. Still, it's only 81 innings in a career of almost 4000 regular season innings. Sandy Koufax was spectatcular in the post-season with a 0.95 ERA in 57 innings, but still, 57 innings in a career of 2300 innings doesn't amount to much. Of course, with the advent of the multiple playoff rounds, starters are throwing more innings in the playoffs, Greg Maddux and Roger Clemens have each thrown 200 post-season innings in careers of over 4500 innings.

And then there's Mariano Rivera. With 889 regular season innings and 112 post-season innings (and a 0.80 ERA) on his resume, one simply cannot ignore the importance of the post-season in evaluating him. With closers, there is not only "game leverage", there is "season leverage". The object is to help one's team win a World Series. If a team is mathematically eliminated or has clinched a division title, and the closer comes on, the leverage may be high vis a vis the game, but it is nil for the purpose of the season. All post-season games have high "season leverage". In this respect, Rivera's 112 post-season innings are not only a significant proportion of his workload, but also disportionately important.

Rivera is, at this point, one of a kind. In the future, though, it does seem likely that post-season performance will be more important in evaluating closers than it is for other players.

A sample reliever's adjusted line

Shall we go back to Tom Henke again. For variety, we will look at his 1986 line, and compare it with Jimmy Key's:

Pitcher IP W W/9 K K/9 HR HR/9 ERA LEV.
Henke-act. 91.3 32 3.15 118 11.63 6 0.59 3.35 1.57 (1.33 adjusted)
Henke-adj. 91.3 35 3.45 107 10.55 7 0.69 3.69 1.57 (1.33 adjusted)
Key 232.0 74 2.87 141 5.47 24 0.93 3.57 1.00(assumed)

I have made the 10% adjustment to Henke's walk, K, HR and ERA totals, as described, and I am using 85% of his leverage in recognition of the strength of the pen overall and Henke's modest leverage. Even with the adjustments, Henke was a better pitcher during the season overall, save for the ERA, but over less than half the innings. Let's see how they fare over their career.

Pitcher IP W W/9 K K/9 HR HR/9 ERA LEV.
Henke-act. 789.7 255 2.91 861 9.81 64 0.73 2.67 1.44 (1.22 adjusted)
Henke-adj. 789.7 281 3.20 775 8.83 70 0.80 2.93 1.44 (1.22 adjusted)
Key 2591.7 668 2.32 1538 5.34 254 0.88 3.51 1.00 (assumed)

Henke was clearly the better pitcher, even with the adjustments. over his career. But, he pitched 1/3 the innings that Key did, and even accounting for the increased leverage (which ought to be adjusted to account for the chaining impact), it is challenging to argue that Henke contributed more than Key did. Henke's ERA+ falls using the adjustments from 156 to 141 (Key's was 122), and applying the leverage factor to his innings results in adjusted innings of about 1,000.  In other words, Henke's career in total had less value than the first 6 years of Dwight Gooden's.

In Part 2, we will compare Tom Henke with the relievers currently inducted in the Hall of Fame, using these measures.

Evaluating Closers- Part 1 | 12 comments | Create New Account
The following comments are owned by whomever posted them. This site is not responsible for what they say.
Magpie - Tuesday, January 23 2007 @ 08:52 AM EST (#162399) #
Henke was clearly the better pitcher... [but] is challenging to argue that Henke contributed more than Key did.

Agreed. Henke pitched better pretty well each and every year, but because Key was providing so many more innings of genuine quality I would have to regard him as the more valuable player in 1985, 86, 87, 88, 91 and 92.

And the Terminator only in 1989 and 1990 when he was just ridiculously, insanely good and Key had fairly ordinary seasons.

Now pause to think about how different those two guys were on the mound: the huge right-hander with the glasses, whose delivery looked like some strange machine unfolding and lurching forward, the medium sized lefty whose every movement was brisk and economical (was anyone else ever reminded of Whitey Ford?) And then enjoy their K-W ratios from 1989:
          K   W
Key    118   27
Henke 116  25

Mike Green - Tuesday, January 23 2007 @ 10:39 AM EST (#162405) #
I just missed Whitey Ford.  I'll have to check out some World Series video from the 1950s.

We'lll get to the meat of this in a week or two, but the essence is that Tom Henke is to Jimmy Key, as Goose Gossage is to Tommy John. Gossage and John had longer careers and pitcher more than their Blue Jay confreres, and with modestly less effectiveness.
Mike Green - Tuesday, January 23 2007 @ 10:57 AM EST (#162408) #
Erm...pitched not pitcher
Pistol - Tuesday, January 23 2007 @ 11:24 AM EST (#162412) #

I was looking at Henke's B-R page and noticed this in the transactions:

January 24, 1985: Chosen by the Toronto Blue Jays from the Texas Rangers as a Free Agent compensation pick.

How did that work?  I don't recall that period.

Henke had an interesting minor league career.  His numbers in the majors were better than the minors and he hit the majors late.  He had a couple cups of coffee and then split time between AAA and the majors at 26 and 27.  I don't remember what the talk was back then but in retrospect he didn't look like that great of a prospect - he's similar to Ryan Houston today.
ayjackson - Tuesday, January 23 2007 @ 11:30 AM EST (#162415) #
Pistol, though I don't have a specific memory of Henke's minor league career or early development years, I would hazard a guess that his success as a pitcher was level-irrelevant and had everything to do with the development of his beautiful split-fingered fastball.
actionjackson - Tuesday, January 23 2007 @ 11:38 AM EST (#162416) #
Back in those days in lieu of compensatory draft picks, you could pick an unprotected player off the roster of the team that signed your type A free agent. When the Texas Rangers signed Cliff Johnson in the '84-'85 offseason, Gillick moved in for the kill. I can't remember how many players the signee was allowed to protect, but it was probably like the Rule V draft in that sense. The Rangers were getting fed up waiting for the big man to come around so they left him off their protected list. The irony is that Heathcliff joined the Jays for the stretch run and hit for a good average with no power, so we got our best closer in history for uh, nothing.
John Northey - Tuesday, January 23 2007 @ 12:26 PM EST (#162419) #

The compensation picks in the pre-1985 period (it was eliminated during the 2 day strike of '85) were a bit more, well, weird than actionjackson listed it.

As I recall, all MLB teams could protect 25 players, but if you signed a free agent you could only protect 24.  This is what the core issue in the strike of 1981 was, believe it or not.  I forget if teams would be able to protect even fewer if they signed multiple free agents, but it was a league wide thing. 

A good example that is linked to the Jays is Tom Seaver.  He was the 26th man on the Mets list after the 1983 season.  The Blue Jays signed Dennis Lamp as a free agent that winter thus giving the Chicago White Sox a compensation pick.  They saw that the Mets were being stupid and took advantage by selecting Seaver after he had a 103 ERA+ season with 231 IP (but a 9-13 record).  He would pitch for 3 more years with an ERA+ above 100 all 3 seasons.  Of course, that opened a slot in the rotation that legend has the Mets putting Dwight Gooden into so it worked out for them (in '85 they just missed the playoffs and Seaver had an ERA+ of 136 while winning his 300th game, bet they wish they protected Seaver instead of George Foster or some other stiff).

actionjackson - Tuesday, January 23 2007 @ 12:59 PM EST (#162426) #
Thanks for the correction John. I believe it occurred in the offseasons of 1982-1985 i.e. between work stoppages and the players union found it to be quite disruptive to their lives so they changed compensation to the sandwich picks we have today. It was a pool of players and the Jays just happened to take Henke from the same team that Johnson signed with. This is I believe the complete list of players taken in this wacky free agent compensation pool draft, the teams that took them and the teams they were taken from:

Joel Skinner  to the Chicago White Sox  from the Pittsburgh Pirates  (1982)
Danny Tartabull  to the Seattle Mariners  from the Cincinnati Reds  (1983)
Steve Mura  to the Chicago White Sox  from the St. Louis Cardinals  (1983)
Tom freakin' Seaver  to the Chicago White Sox  from the New York Mets  (1984)
Tim Belcher  to the Oakland A's  from the New York Yankees  (1984)
Tom Henke   to the Toronto Blue Jays  from the Texas Rangers  (1985)
Donnie Moore  to the California Angels  from the Atlanta Braves  (1985)
Angel Salazar  to the St. Louis Cardinals  from the Montreal Expos  (1985)

The White Sox profited greatly from this pool as they also held the Cubs for ransom in 1983 when the Cubs left Fergie Jenkins unprotected. They got Pat Tabler, Scott Fletcher, Randy Martz, and Dick Tidrow in return for Steve Trout and Warren Brusstar with a promise to not take Jenkins with their pick. They then turned around and dealt Tabler to the Indians for Jerry Dybzinski. OK, they didn't make out as well as they could have, but they did win 99 games that year under TLR. If anyone has any players that I missed please feel free to fill them in.

Mike D - Tuesday, January 23 2007 @ 01:33 PM EST (#162433) #

"Season leverage" is a fascinating concept, Mike G.  It's surely the highest of high-leverage situations for any player, and is therefore potentially incredibly useful in assessing players' value.  However, because postseason stats aren't included in career totals, many Hall of Fame arguments and analyses don't include playoff performance at all! 

Retrosheet is making a lot of fascinating studies possible.  I wonder if there could be some sort of "Win Shares Enhanced" where playoff and pennant race regular season games could factor in "season leverage."   It would be a really useful exploration of "clutch" -- rather than using assumptions about whether it's possible for a player to be the type of person to thrive in key situations, you can measure what the player actually contributed when the leverage was highest.

MatO - Tuesday, January 23 2007 @ 01:47 PM EST (#162437) #
Legend has it that Henke was spotted pitching in the winter leagues by the Blue Jays.  He had added a splitter to his hard fastball and mediocre slider and it was the splitter that completely changed him as a pitcher.  Henke was not a favourite of then Texas manager Doug Rader and was left unprotected in that weird draft and the Jays happily took him.  This is what the Jays did the best under Gillick, finding hidden talent.
Craig B - Tuesday, January 30 2007 @ 04:24 PM EST (#162822) #

Mike mentioned Mariano Rivera's postseason performance as a significant component of his career, but omitted the fact that Rivera's 0.80 ERA in the playoffs makes each postseason inning about twice as valuable (as well as having many times more leverage) as each of his regular season innings, even though by his regular season innings he easily grades out as the best per-inning pitcher of all time. 

Rivera's eight postseason wins tie him for 11th on the all-time list, by the way, and he has more than twice as many postseason saves as the #2 man, Dennis Eckersley (34-15).  He also has more postseason wins than any other pitcher with zero or one postseason loss.

He's so far out in front of the other great pitchers (except Pedro, who may be coming back to the pack) that it's mind-boggling and it's why I consider him the greatest pitcher of all-time.

Evaluating Closers- Part 1 | 12 comments | Create New Account
The following comments are owned by whomever posted them. This site is not responsible for what they say.