Bayesian Workflow of a Situational Random Effects Model
Abstract
This paper describes in detail the Bayesian workflow for fitting a multilevel model to balls-in-play data where there is a bias term to account for a count advantage. All of the steps of Bayesian model fitting are described including specifying a model script, obtaining a matrix of simulated draws from the posterior, summarizing the draws to perform inference, and generating replicated predicted datasets to check the adequacy of the model. Different methods for fitting random effects models such as maximum likelihood, JAGS and Stan are compared.
1 Introduction
A general problem in baseball is understanding the sources of variation of outcomes of balls put into play. This paper describes the application of a Bayesian approach for modeling balls in play with a focus on predicting outcomes in future seasons.
The first step of this Bayesian modeling is determining a suitable measure of the outcome variable. One could focus on a wOBA measure where each outcome on a ball in play is assigned a suitable weight. But the result of a ball in play depends not only on the quality of the ball hit off the bat, measured by the Statcast launch variables, but also on the luck component where hard-hit balls can be outs and softly-hit balls land for hits. A better measure of batter performance is the expected wOBA which provides an estimate of the wOBA given values of the launch angle and exit velocity.
The next step in the construction of a Bayesian regression model determines the important inputs that explain much of the variation of the outcome variable. What happens on a ball put in play depends primarily on the batter rather than the pitcher so the model should include batter ability effects. Also since the outcomes of balls in play can depend on particular situations, it is desirable to include terms in the model that allow for the situation. One of the most important situations determining the outcome on a ball in play is the count, and the regression model will include a term indicating if the count is neutral (like a 1-1 count) or if the count favors the pitcher (like a 0-2 count).
Here are some goals of this modeling exercise:
Provide good estimates of the batter abilities to be productive on balls put in play. These estimates will shrink or adjust the observed averages towards the overall mean and are better than the observed averages in predicting future performance.
Estimate the situational effect. In this case, we want to measure the advantage that batters have when hitting in a neutral count relative to a pitcher’s count.
Provide predictions of future performance. In addition, by simulating replications of the current data from the fitted model, one can check if the model is a suitable fit to the observed data.
Section 2 describes the Statcast data that is used in this study and Section 3 illustrates the use of a mean-difference plot to see how batters take advantage of a favorable count. Section 4 describes the multilevel model for batter abilities with a bias term accounting for the count. Section 5 provides an overview of the use of one particular MCMC simulation algorithm (JAGS) to simulate from the joint posterior distribution. The output of JAGS is a matrix of simulated draws from the posterior and Sections 6 and 7 illustrate the use of this simulated parameter matrix for inference and prediction. In the concluding Section 8, we talk generally about situational effects in baseball and give an overview of alternative methods for fitting these multilevel models. Although the focus of the study is on the use of JAGS, the Appendix gives details on alternative methods for fitting the model using maximum likelihood and Stan.
All of the work in this paper is reproducible. The Quarto file at https://gist.github.com/bayesball/5877d9c2239f48189b0ed4c1dfe3167a reads the Statcast data from a Github site, fits the model using JAGS, and implements all of the inferential and prediction calculations. R code for reproducing all of the figures in this paper is provided in this file.
2 Data
Statcast data is collected for all games in the 2023 season before July 15. We focus on balls in play and the response variable is the expected wOBA which is the estimate of the wOBA based on the launch angle and exit velocity measurements. We also collect the identity of the batter and the balls and strikes count. From the hitter’s perspective, the count is categorized as “ahead” (2-0, 3-0, 3-1), “neutral” (0-0, 1-0, 1-1, 2-1, 3-2), or “behind” (0-1, 0-2, 1-2, 2-2). Since 94% of the balls in play occur on neutral or behind counts, we remove the ahead-in-count balls in play from the data and focus on the batter’s advantage in a neutral count compared with a behind count.
In this paper, we focus on the square root of the expected wOBA \[ rwOBA = \sqrt{E(wOBA)} \]
The root reexpression is taken primarily for modeling considerations. The top panel of Figure 1 displays a histogram of the expected wOBA variable. Note that this variable is heavily right-skewed. The bottom panel displays a histogram of the square root of \(E(wOBA)\). Note that this transformed variable is more bell-shaped which is consistent with the modeling assumption (to be described shortly) that the sampling errors are normally distributed.
3 A Mean-Difference Plot
We collect data for all batters who had at least 100 balls in play before July 15. Since we are interested in understanding the batter advantage due to the count, we compute for each player the mean root wOBA on a neutral count and a mean root wOBA on a behind count:
\[ \bar Y_n = \frac{\Sigma \, rwOBA_{neutral}}{BIP_{neutral}}, \bar Y_b = \frac{\Sigma \, rwOBA_{behind}}{BIP_{behind}} \]
From this paired data, we construct a Tukey mean-difference plot (Kozak and Wnuk, 2014) in Figure 2 which graphs the average \(M = (\bar {Y_{n}} + \bar {Y_{b}}) / 2\) against the difference \(D = \bar {Y_{n}} - \bar {Y_{b}}\). The average \(M\) measures the general ability of the batter to produce on a ball in play, and the difference \(D\) measures the batter’s advantage in hitting on a neutral count compared to a behind count. Note that there is a wide scatter in the mean (horizontal) values of \(M\), indicating that there are significant differences among the players in the quality of the balls put into play. A majority of the points fall over the horizontal line at 0, indicating that players tend to take advantage of the neutral count with a positive difference \(D\). But there is substantial variation in the values of \(D\) ranging from \(-0.1\) to \(0.2\).