Batter's Box Interactive Magazine Batter's Box Interactive Magazine Batter's Box Interactive Magazine

Nobody’s been that glad to get out of Denver since Neal Cassady.

I had planned to give Jays fans a nice scouting report of some Rockies players and a recap of how Toronto hitters looked at Coors Field with today’s series report. However, the Sportsnet Gods conspired against me by not showing this series on Sportsnet and left me unable to watch any of the three games. Subsequently, I only got to listen to portions of the three games on the radio, including only the last three innings of yesterday’s match. All in all, perhaps I should be thankful for that considering the way Toronto played in Denver.

So, let’s turn our attention to win probability added (WPA), which is being tracked daily this season by the wonderful Fangraphs website. Let’s look at what WPA has to tell us about the Blue Jays bullpen this season.

Before we begin, you should familiarise yourself with WPA. WPA is like any other statistic. It has its uses and its limitations. Just as one wouldn’t to use saves to determine which of two ballplayers is likely to be better the next season (unless your information was extremely limited), one wouldn’t want to use WPA to predict that, either. Dave Studeman is a leading proponent of the statistic and has published several articles over at The Hardball Times on WPA and its uses. Particularly illuminating for the beginner is this primer that Studeman wrote in 2004. As he stresses, WPA has its uses and its shortcomings. It is not a predicative tool. Just because Vinnie Chulk is the worst pitcher on the Jays right now, it doesn’t mean he is necessarily likely to be the worst pitcher over the course of the season.

However, one area WPA is very illuminating is in evaluating bullpen effectiveness and strategy. As Studeman writes, “Using the [Win Expectancy] Finder again, if a pitcher gives up a bases-empty home run in the first inning of a tie game, his team's Win Probability decreases about 10%. If he does the same thing in the eighth inning, it decreases about 25%, because his team has less time to come back. In this context, the eighth inning is about 2.5 times more important than the first inning. And if you apply this sort of analysis to every appearance made by a relief pitcher, you can quantify the importance of all of his innings pitched.”

After the ballgame is over it doesn’t really matter when the team scored a run, because the run counts the same. However, during the game a run in the second inning is very different from a run in the eighth inning. Think about how differently yesterday’s game would have been managed if Rios had not hit his 3-run homer in the fifth inning, but instead had hit it with two outs in the ninth inning. Not only would the Jays had only one out to work with when they needed two runs, but it would have affected how both coaching staffs managed the game and we likely would have seen different pitchers on the mound, different strategies employed and so forth.

WPA can work well as a measure of a reliever’s effectiveness because many traditional statistics, such as ERA, are inadequate as tools of evaluation. Relievers pitch so few innings that a disastrous inning can have a disproportion effect on their ERA. I care little if Scott Downs allows 6 runs in a game Toronto is losing 8-2 to the Indians, but this could raise his ERA by a whole run at the end of the year. Similarly, ERA doesn’t reflect inherited runners at all, which is one of a reliever’s most important duties. A reliever with a 2.40 ERA who allows 75% of inherited runners to score isn’t doing a very good job, despite his low ERA. Just like Leverage Index, WPA reflects the fact that some relieves pitch when the game is on the line, and their statistics can’t be treated identically to those of starters. WPA allows analysts (including amateurs like myself) to actually quantify what a reliever contributed (or deducted) to a team’s chance of winning the game.

(Note: These stats are as of before Sunday’s game.)








B.J. Ryan







Justin Speier







Jason Frasor







Brian Tallet







Scott Downs







Francisco Rosario







Scott Schoeneweis







Shaun Marcum







Dustin McGowan







Pete Walker







Vinnie Chulk







This table reveals a lot of information that we knew or could have guessed, but it also reveals some interesting statistics. To begin with, one number dwarves all the others, and that’s BJ Ryan’s 136.9. That number is the largest positive contribution on this year’s Jays team, although Josh Towers’ -178.3 is the biggest absolute number. It’s no small feat that Ryan has, according to WPA, won Toronto one and one-third games by himself so far this season. Ryan also has the very impressive statistic of being the only Jay to have not had an appearance where he posted a negative WPA score. That is, every game he’s pitched in he has contributed positively to Toronto’s chances of winning the game. He’s been the third-best reliever in the AL so far this season, behind only Jonathan Papelbon (191.8) and Chris Ray (147.0). Even with Mariano Rivera’s struggles so far this season, a team doesn’t want to enter the ninth inning losing to a team in the AL East. The odds for a comeback are slim, at best.

Now, that’s not a surprise. BJ Ryan is a beast. However, some of you may be wondering what the story is with Brian Tallet? How can a 9.00 ERA in 7 innings have made him fourth-best in the Jays bullpen in WPA? Simply put, in the only game he pitched where it really mattered he did well. Tallet had three positive appearances and three negative ones. To start with the best, on April 8th Tallet pitched 2.2 scoreless innings against Tampa Bay in an 8-4 win. He gained +15.7 for the outing, as Toronto came back to win after falling behind in a forgettable Ted Lilly start. Tallet got the win as Toronto rallied while he was pitching and Tallet worked very effectively, striking out three while allowing two hits. Tallet also added positive contribution on April 14th when he pitched a 2-run inning against the White Sox in a 13-7 win. He gained +0.1 for the appearance, because the odds of Toronto winning were so large that allowing 2 runs in one inning didn’t matter. On April 30th Tallet pitched a scoreless inning against the Yankees in a 4-1 loss. He gained +0.7 for the appearance.

Tallet had three negative outings, but these also came in games where he was pitching very low-leverage innings. On April 5th Tallet pitched 1.1 two-run innings against Minnesota in an 13-4 win. He ended up with -1.5 for the outing. He also gave up two runs on April 13th, where he was only penalised -0.1. Toronto beat Boston 8-6, but Tallet entered the game in the top of the eighth inning, when it was 8-1. This was the game where Frasor gave up 3 in the ninth, before Ryan shut the door.

Tallet gave up a homer to Miguel Tejada on May 1 against the Orioles that scored two runs (one his, one Schoeneweis’s), but this was in the ninth inning of a game that the Jays were winning by four. The Jays held onto win by 2 and Tallet was penalised with only a -1.1. MGL spoke about a similar situation in a thread over at BTF yesterday, discussing Wagner’s Saturday implosion, as he argued against using Wagner in that situation, unless there was a specific purpose to pitching him that particular day. He argued that the Mets stood about a 98.1% chance of winning the game with a really bad pitcher pitching (say, Jose Lima), as compared to 99.2% with Wagner. I’m unsure if that takes into account the quality of the opposition, which WPA doesn’t, but it does demonstrate that this homer was relatively meaningless. Even if he had stayed in to finish it, Tallet would almost certainly have won the game. And thus concludes the story of how Brian Tallet has been the fourth-best reliever on the Jays so far this year.

So now the mystery of Brian Tallet is solved, onto other things. Justin Speier is second to B.J. Ryan, which is not a surprise, although I didn’t think that Jason Frasor would be this close to him. This is a reflection of both the usage pattern of Speier and Frasor, as well as how they have pitched. Both are being used in high-leverage situations and, more often than not they succeed. Seeing Chulk with a negative score is also not surprising, as he’s been at the centre of a few collapses in high-leverage situations, which was bound to outweigh his positive contributions. But, let’s investigate the difference between Chulk and Frasor in more detail.

Frasor’s made 12 appearances, of which he’s given up runs in half of them. He allowed four runs in the second game of the season, the aforementioned 13-4 loss to the Twins. However, given the insignificance of those runs, he was only docked -3.7 for that game. In another previously discussed game, Frasor gave up 3 runs to the Red Sox following a strong Lilly start and the Jays held on to win 8-6. This one only cost him -0.9, as three runs in the ninth inning with a sizable lead costs you very little when you also get two outs at the same time. Frasor gave up 2 runs to the O’s on April 27th, when the Jays lost to Baltimore 8-6, which cost him -13.2. That was his most costly outing of the season, as he turned a 4-2 Baltimore lead into a 7-2 margin, by allowing an inherited runner and two of his own to score. Frasor gave up one run thrice, but actually ended up with positive scores in two of the games. Most of these outings, besides the April 7th and 27th cost him less then -4 points. Frasor’s had only one noticeable poor appearance.

Most of his positive appearances balance that out, with the highlights being 8.3 on April 8th and 8.2 on April 26th; the latter where he pitched the sixth and part of the seventh inning in a game Toronto was leading 3-1 at the time. His most recent outing is what accounts for nearly his entire positive score, as he received 35.3 points for his great job against Anaheim on Thursday evening. In the bottom of the ninth of a tied game Frasor managed to escape from a situation where a runner was on third with one out (the out having been recorded on the previous batter by Scott Downs) and managed to force a 10th inning. He then retired the side 1-2-3 in the 10th, although Toronto’s had a 4-run lead at that point.

That one game is currently the defining moment of Frasor’s season. So what performance defines Chulk’s 2006? Vinnie’s had four games where he’s amassed WPA scores higher than 10 or lower than -10. In his one positive performance he entered the April 14th contest against the White Sox in the sixth inning, with Toronto leading 7-5, and pitched three scoreless innings as the Jays added four runs. For those three innings he had a 10.2 WPA score, which was the highest score amongst Jays’ pitchers for that game.

Chulk’s three negative performances demonstrate some of the strengths of WPA. On April 5th Chulk received -10.7 in a game Toronto would eventually lose 13-4. The game was only 5-4 for the Twins when Chulk entered, although Tallet had left runners at first and third. Chulk allowed both runners and one of his own to score in the inning, and turned the game into one where Toronto had little chance of coming back. No other statistic can recognise the fact that Chulk allowed these runners to score quite like WPA, as even inherited runner percentages still can’t take into account outs and game situation. On April 23rd Chulk entered the game in the top of the eighth with the Red Sox leading 4-3, and promptly surrendered a walk and two singles. He left the now 5-3 ballgame with runners on first and second with none out and Ryan came in and didn’t allow Youkilis or Loretta to score. Again, WPA captures exactly how costly Chulk’s performance was, as Ryan really saved his stat line that game, but that does not ignore the fact that Chulk created this situation and only Ryan’s exemplary performance kept the Jays in the game.

In Chulk’s most costly performance – the one that may define his season so far – he was penalised -27.5 WPA on the April 7th game against Tampa. He was summoned by John Gibbons in the sixth to preserve the 6-3 lead and Scott Downs’s victory and he allowed singles to Johnny Gomes and Aubrey Huff before Ty Wigginton hit a three-run homer to tie the game. Although Chulk escaped the inning without any more runs, the damage was done. Pete Walker would allow a 2-run homer in the seventh to give Tampa a lead they would not relinquish. In fact, Chulk and Walker’s appearances in this game are tied for the most costly relief appearance by a Blue Jays reliever so far this season.

In conclusion, what we can learn from WPA is that Ryan is incredibly dominant; Tallet’s been the Jays fourth-best reliever; McGowan is incredibly costly on a WPA/inning basis; Frasor’s ERA is mostly the result of a few bad performances in games that really didn’t matter much and that Chulk’s poor outings are much more costly.

Now, this doesn’t mean very much going forward, as one good performance by McGowan next time he’s in Toronto could erase his whole deficit. What is likely to continue is that Ryan, Speier and Frasor will continue to get the high-leverage innings, with Schoeneweis facing a left-handed batter or two. Not only have the three been used that way so far this year, but they’ve performed well and have given Gibbons no reason to change his usage of the bullpen so far. Gibbons has employed some “unorthodox” usage of Ryan, which I support as long as he continues to be careful about it. However, as progressive as Gibbons may be with Ryan, he’s still cognisant of the save statistic, as the chart below demonstrates. Nevertheless, this may be the best approach, if it’s what makes Ryan the most comfortable, as long as Ryan is willing to pitch more than one inning on some occasions and as long as Speier and Frasor continue to pitch well in the middle innings.

Stay tuned until the next WPA bullpen update….

Now, for your reading pleasure:

Top 5 Starts

  1. Casey Janssen, 48.7, vs. Angels, May 17
  2. Casey Janssen, 46.5, vs. Angels, May 7
  3. Roy Halladay, 36.9, vs. Angels, May 8
  4. Josh Towers, 33.1, vs. Devil Rays, May 14
  5. Roy Halladay, 31.7, vs. Yankees, April 28

Top 5 Relief Appearances

  1. Jason Frasor, 35.3, vs. Angels, May 18
  2. Justin Speier, 29.8, vs. Red Sox, April 21
  3. BJ Ryan, 29.8, vs. Red Sox, April 21
  4. Justin Speier, 27.0, vs. A’s, May 10
  5. BJ Ryan, 21.9, vs. Red Sox, May 3

A Denver Disaster | 11 comments | Create New Account
The following comments are owned by whomever posted them. This site is not responsible for what they say.
Jim - Monday, May 22 2006 @ 11:05 AM EDT (#147341) #

While I find WPA interesting, the fact there is no way to quantify the defense makes it tough to take too seriously.  For example, a pitcher gets 100% of the credit if an outfielder scales a fence and steals a home run.   A batter gets 100% credit for hitting a fly ball that falls between 2 fielders who are staring at each other waiting for the other to make the play. 

I know these things tend to work out in the aggregate, but they don't always.


Magpie - Monday, May 22 2006 @ 11:28 AM EDT (#147342) #
As it turned out, it wasn't a Lyle Lovett weekend. Mr Bob Seger:

I still remember it was autumn and the moon was shining
Our '60 Cadillac was roaring through Nebraska whining
Doing 120 man the fields was bending over
Heading up for the mountains knowing we was traveling further
All the fires were blazing and the spinning wheels were turning turning
Had my girlfriend beside me, brother, she was burning burning

I walked up back to speak to this southern funky school teacher
She had a lot of somethig heavy but we couldn't reach her
We told her we needed something that would get us going
She pulled out all she had and laid it on the counter showing
All i had to do was lay my money down and pick it up
The cops came busting in and we lit out in a pickup truck

Go! get out of Denver, better go
Get out of Denver, better go
Get out of Denver, better go
Get out of Denver
Cause you look just like a commie
And you might just be a member
Better get out of Denver
Better get out of Denver...


Mike Green - Monday, May 22 2006 @ 11:43 AM EDT (#147343) #
Large scale studies have shown that clutch hitting does not vary much from overall hitting ability. As Thomas suggests, WPA is not much of a predictive tool, but it is fairly descriptive of what has occurred.  The bloop single in a ninth-inning tie game will attract more WPA than a 500 foot homer with a 2-0 lead in the fourth because it happened to be more important at the time.

On a completely different note, the decision to give Hinske more work in Colorado will pay dividends down the road. 

Geoff - Monday, May 22 2006 @ 01:16 PM EDT (#147350) #
As Thomas suggests, WPA is not much of a predictive tool, but it is fairly descriptive of what has occurred.

I'm curious if there are any baseball stats that would be considered a predictive tool rather than a descriptive tool.

HR, OPS, SB, IP, WHIP, WS, ERA+  <— Which of these are predictive?
King Ryan - Monday, May 22 2006 @ 02:51 PM EDT (#147351) #
Strikeouts and walks tend to be considered the most predictive of statistics.  This is why, when the Batter's Box looks at minor leaguers and prospects, the author tends to make mention of how well he's controlling the strikezone (i.e. K/BB ratio)  These players tend to have a better chance of succeeding in the majors than players who strikeout lots and don't walk, even if the latter player has nice numbers otherwise (Hello, Mr. Phelps)
Craig B - Tuesday, May 23 2006 @ 09:06 AM EDT (#147368) #
Why would Durazo want to play for Toronto instead of the Yankees?  He's got a much clearer road to the majors with New York.
Ryan Day - Tuesday, May 23 2006 @ 12:14 PM EDT (#147391) #
  Never mind that Durazo was hitting just 289/365/434 in AAA, in the PCL, even. Decent, sure, but not better than Hillenbrand has been.

  Other guys the Jays apparently should have acquired to DH:

 Frank Thomas: 195/310/431
 Hee Seop Choi: 268/424/438 (AAA)

  As much as I may have agreed it would have been a nice idea to try one of these guys, for the most part they're not even outhitting Eric Hinske, never mind Shea Hillenbrand. I think it's safe to say at this point that keeping Hillenbrand was the right move.

TangoTiger - Wednesday, May 31 2006 @ 12:56 PM EDT (#148044) #
Jim: an outfielder scales a fence and takes a sure HR away from Wells, leaving Wells with an out. Does that mean you can't take the number of outs that Wells hit "too seriously"? Or a ball bounces off Jose Canseco's head, for a HR to the hitter. Why should the hitter get credit for a HR? Now, as for your contention that there is "no way to quantify defense", you are off-base. There is a way to quantify defense. It's just harder, that's all. As you surmised, things tend to work out in the aggregate. I once published the 1999-2002 WPA, and I calculated it two ways for balls in play: give 100% of the credit to the pitchers, and give 100% of the credit to the fielders. Result? Virtually no change. (DIPS-disciples would have guessed this.) Win Expectancy and Leverage Index quantify exactly what is happening exactly as you are watching it. It's a pulse-monitor. The better data you can feed the system, the better you can apportion the impact to the players involved.
A Denver Disaster | 11 comments | Create New Account
The following comments are owned by whomever posted them. This site is not responsible for what they say.