Scaling Stan's performance with `reduce_sum` in practice

While Stan is awesome for writing models, as the size of the data or complexity of the model increases it can become impractical to develop iteratively a model due to too long execution times. The new and shiny parallelization facility `reduce_sum` released with Stan 2.23 makes it easy for users to scale the performance of Stan programs with more CPU cores. `reduce_sum` is designed to calculate in parallel a (large) sum of independent function evaluations such as the sum of independent likelihood terms for the observed data as applicable to most Stan programs (GP problems would not qualify though). In this talk we will provide lessons learned from experience with the new `reduce_sum` facility. This includes as to how we recommend to switch an existing Stan program in steps over to use `reduce_sum`. We will cover what aspects users need to consider for good scalable performance and how common coding errors can be avoided. We will demonstrated the steps using COVID-19 transmission models developed by the Imperial College London COVID-19 response team. Their model inference times reduced from days to hours and as a result enabled real time fitting of Bayesian semi-mechanistic models to COVID-19 pandemic data.


Presenter biography:
Sebastian Weber

Dr. Sebastian Weber is working as Director of Statistical Methods and Consulting in the pharmaceutical industry at Novartis. He has extensive experience in leveraging historical control data in clinical trials, designing Oncology phase I dose-escalation trials and is involved in pediatric drug development, where he applies extrapolation concepts. His research interests include the application of pharmacometric principles in statistics, model-based drug development and use of Bayesian methods in drug development. Sebastian supports the Stan project as a developer with focus areas on ordinary differential equation solvers and within-chain parallelisation.