Starters and relievers: What if their roles were reversed?

What would happen if Aroldis Chapman were to start? Or if Max Scherzer were to close? It's a strange invisible fence that seems to separate the roles of starter and reliever. If the Nationals wanted to, they could use Scherzer in relief. The Reds would have to stretch Chapman out a bit, but they could choose to make him a starter. And the world would continue spinning. It's a maddening question, but despite the constant pleas of the listenership of Effectively Wild, no team seems willing to run the experiment that would answer it. Pitchers either pitch seven innings or they pitch one. (Unless they pitch for Tampa Bay.)

We kind of have an idea of what would probably happen. In fact, we have a tiny bit of data. Henry Druschel recently looked at how starters fared when sent to the 'pen in the playoffs. The results were a little confusing (the sample size was also small), in that starters-turned-relievers gave up more walks, struck out fewer batters ... and gave up fewer runs than they did in the regular season. Weird.

The general consensus is usually that as a starter, a pitcher has to pace himself a bit because he'll (hopefully) be out there for six or seven innings and he'll have to turn the lineup over twice. As a reliever, he'll be out there for one inning and he can just let it fly. Relievers, in general, have higher strikeout rates than their starting counterparts. It makes sense. If you don't have to pace yourself, reaching back for that extra gear is something you can do against every hitter you face. All four of them. Turn a starter into a reliever and he'll turn into a monster.

Teams also seem to have made up their minds about which job is harder. The top starters in baseball make upwards of $25 million per season. The top closers might get $15 million. And there's something to be said for the fact that there are plenty of guys around who are former (failed) starters who have found a second life as a bullpen arm, while there aren't that many former relievers in rotations today. Although relievers have better numbers, it seems starting is actually a harder gig. So, we should expect an ace starter to be the best closer ever if he were put into that role, and perhaps an ace closer to be a good starter.

But from a research perspective, we're missing something. We know how relievers and starters seem to perform differently, but it's still two entirely different groups of guys in each basket. What if Chapman, by having to pace himself, would completely lose what makes him amazing? What if Scherzer the closer would still be a good pitcher, but the pacing thing didn't really make much of a difference? We need to find some way to look at what happens to the same guy (or set of guys) when they are starting and when they are relieving.

Warning! Gory Mathematical Details Ahead!
We won't be able to get a good sample of players who start and relieve in the same season. There are a few, of course, but we also have to realize that a good way to "become" a reliever is to get hurt (are we seeing the effects of a transition to relief, or are you just hurt?), to get demoted (are we seeing improvement because you hit some bad luck as a starter and are now regressing to the mean?) or to simply have the calendar year change over and the team's needs change (are you the same person you were last year?).

Instead, I'd suggest we look at it a different way. Relief appearances come in different shapes and sizes. While the standard relief appearance these days is one inning or less, there are times when a reliever is asked to go a little deeper. Sometimes, he's the long man that night. Sometimes, the manager just wants him to get three innings to finish off a game. In those outings, our reliever does not need to pitch six innings, but he will have to pace himself and potentially think about turning the lineup over a bit.

Using data from 2003-14, I looked for all relief appearances in which the pitcher faced more than 12 batters. It's possible that in a few of those, he didn't go out there expecting to face 12 batters, but for the most part, he knew what he was getting into. I also looked for all appearances in which a reliever faced six batters or fewer. It's likely then that he knew he'd only be out there for an inning or so and he could just let it go. In this way, we can at least see what pacing does to a pitcher's performance. It's not a perfect match for starting, but it does have some of what we're looking for. I looked for pitchers who had at least five appearances of both kinds within a year.

Using the log-odds method, I calculated the chances that, based on the pitcher's seasonal stats and those of the batter he was facing, a plate appearance would end in a strikeout. (I later did the same for walks, singles, etc.) I used this as a control variable within a binary logistic regression, and then coded for whether the plate appearance took place during an outing of fewer than six batters or one of 12 or greater. (Plate appearances with six to 11 batters faced were not included.) If a pitcher was more likely to strike a batter out in a short appearance vs. a long one, we'd see it in the long vs. short variable.

The results? When a pitcher in this sample was pitching in a short appearance, his strikeout rate was much higher, on the order of about 1.5 percentage points for the average pitcher in the sample. That was the only finding that was statistically significant among the outcomes. Walks were marginally so (p = .147 ... which, yeah, that's stretching it), but that arrow also pointed toward more walks in a short outing for the same pitcher. Singles went down in short outings (p = .120, so again, take that for what it's worth), and overall OBP was not affected, nor were extra-base hits or home runs. And while overall BABIP wasn't affected, the chances that the ball would be put into play were significantly impacted. When a pitcher was pitching a longer outing, significantly more of the batters he faced put the ball in play.

What's fascinating is that when a reliever had to pace himself, he was clearly using a different, pitch-to-contact approach, but overall results weren't affected much. The way in which he got outs changed somewhat from strikeouts to groundouts and pop flies, and the way he put runners on first changed from walks to singles, but the results were just different. Not better or worse.

Now, the question is usually asked about the elite pitchers. Scherzer and Chapman. The guys in this sample are bouncing around roles in the bullpen, probably picking up whatever work happens to be available. They're probably not elite. They're probably doing that job not because the manager thinks they are perfect for the gig, but because the team needed warm bodies to go out there and do it. The idea of pitching to contact makes sense given the demands of the assignment, so maybe it's just reasonable professionals doing what needs to be done. Interpret with care.

These data suggest that someone like Chapman would lose some of his strikeout mojo in the rotation (and, of course, the reason that people are obsessed with him to begin with is because of the strikeout mojo). Scherzer might pick up a little on his K rate, but he might also issue a few extra walks. He might be a more electric pitcher, but that's not always the same thing as being a better one.

We'll Never Know ...
Of course, other than the occasional Game 7 stint on the mound for an ace pitcher, we'll never have a big enough data set to really answer this question. Still, I think we have a bit of direction and a bit of warning on something to watch out for when we theorize on the matter. I think that line of reasoning when people think about this goes something like:

1. Chapman is good. (True.)

2. Chapman strikes out a lot of guys. (True.)

3. Chapman is good because he strikes out a lot of guys. (There is, of course, some truth to this, but there's more to pitching than just strikeouts.)

4. If Chapman started, he might lose some of those strikeouts. (Probably.)

5. This means Chapman wouldn't be as good. (Maybe, maybe not. He might lose some K power, but what if he also stops walking more than four batters per nine?)

Chapman would still have an electric arm, and even if he had to sit lower in velocity to make it through six innings (perhaps throwing 96 mph on a regular basis), he'd still use velocity as a key part of his game plan. Maybe he'd use it in a slightly different way, aiming for weak contact, rather than just a swing and miss. But does Chapman have the skills to do that consistently. If he's not throwing "you can't touch this" stuff, but merely 96, can he hit his spots to actually get the swing and the not-quite-on-the-sweet-spot contact? Maybe.

It's good to remember that when we do these sorts of analyses and then project our conclusions onto people who weren't actually in the data set, we need to be careful. Pitching coaches and managers are essentially paid to introduce bias into our data sets. When we study fielding at third base, we're looking at the 30 or so guys whom MLB managers thought were the best option at third base. If we assume that managers have some idea of what they are doing, we shouldn't quickly assume that this second base guy over here would do the same if he were moved to third. In the same way, maybe the reason Chapman isn't a starter right now is that the Reds honestly believe that he'd be a bad one, or at least a mediocre one, and that his value as a closer would eclipse his value as a starter. It's always possible that they are wrong, but don't be quick to assume that.

To answer the question at hand, the data here give us some idea of what to expect. If Chapman and Scherzer switched roles, they'd probably pitch a little differently than they do now on account of the role. And that would have an effect on them and the results would take a different form. But the best guess that we can make is that, overall, they'd be about as effective as they are in their current role.