Effortless frequentist covariances of posterior expectations in Stan
The frequentist variability of Bayesian posterior expectations can provide meaningful measures of uncertainty even when models are misspecified, but classical Gaussian approximations based on the maximum a posteriori estimate can be singular or difficult to compute. These difficulties have prompted some recent authors have turned to bootstrapping Markov Chain Monte Carlo (MCMC) procedures. Though naively parallelizeable, the bootstrap remains extremely computationally intensive, especially for long-running MCMC procedures. We introduce the Bayesian infinitesimal jackknife (IJ), a consistent estimator of the frequentist covariance of an MCMC posterior expectation which is easily computable from a single MCMC chain. The bulk of the IJ's required computation is equivalent to what is required for the LOO package, so the IJ can already be computed automatically for rstanarm models, or from generic Stan models using existing, well-documented techniques. We provide an R package to compute the IJ from rstanarm and Stan output and demonstrate the IJ's accuracy and computational benefits by simulation and by comparison with the bootstrap on models taken from the Stan examples collection.
Ryan Giordano is currently a Statistics Postdoctoral Associate in Tamara Broderick’s group at MIT. Ryan received his PhD in statistics from UC Berkeley, where he was advised by Michael Jordan, Tamara Broderick, and Jon McAuliffe. Ryan has additionally earned an MSc with distinction in econometrics and mathematical economics from the London School of Economics and undergraduate degrees in mathematics and theoretical and applied mechanics from the University of Illinois in Urbana-Champaign. He has worked as an engineer for Google and HP and served as an education volunteer in the US Peace Corps in Kazakhstan. His research interests include variational methods, Bayesian robustness, sensitivity analysis, open software, and an amateur interest in the philosophy of science.