Handbook of Markov Chain Monte Carlo

Edited by Steve Brooks, Andrew Gelman, Galin L. Jones and Xiao-Li Meng

Since their popularization in the 1990s, Markov chain Monte Carlo (MCMC) methods have revolutionized statistical computing and have had an especially profound impact on the practice of Bayesian statistics. Furthermore, MCMC methods have enabled the development and use of intricate models in an astonishing array of disciplines as diverse as fisheries science and economics. The wide-ranging practical importance of MCMC has sparked an expansive and deep investigation into fundamental Markov chain theory.

The Handbook of Markov Chain Monte Carlo provides a reference for the broad audience of developers and users of MCMC methodology interested in keeping up with cutting-edge theory and applications. The first half of the book covers MCMC foundations, methodology and algorithms. The second half considers the use of MCMC in a variety of practical applications including some in educational research, astrophysics, brain imaging, ecology and sociology.

The in-depth introductory section of the book allows graduate students and practicing scientists new to MCMC to become thoroughly acquainted with the basic theory, algorithms and applications. The book supplies detailed examples and case studies of realistic scientific problems presenting the diversity of methods used by the wide-ranging MCMC community. Those familiar with MCMC methods will find this book a useful refresher of current theory and recent developments.

Reviews

Markov chain Monte Carlo (MCMC) is a family of algorithms used to produce approximate random samples from a probability distribution too difficult to sample directly. The method produces a Markov chain that whose equilibrium distribution matches that of the desired probability distribution. Since Markov chains are typically not independent, the theory of MCMC is more complex than that of simple Monte Carlo sampling.

MCMC was invented in 1953 but was not widely applied until statisticians became aware of it around 1990. Then problems that had previously been intractable suddenly became tractable. MCMC worked like magic. (We will return to the comparison with magic below.)

One of the most popular books on MCMC to date has been Markov Chain Monte Carlo in Practice. This book was edited by Gilks, Richardson, and Spiegelhalter and published by Chapman & Hall in 1995, just five years after the use of MCMC became wide-spread among statisticians. Now Chapman & Hall has published a new book Handbook of Markov Chain Monte Carlo, edited by Brooks, Gelman, Jones, and Ming. The Handbook is in some sense an update to MCMC in Practice reflecting 16 years of theoretical development and experience using MCMC.

After two decades of widespread use, MCMC remains magical, both in the sense of being powerful and in the sense of being mysterious. As Charles Geyer explains in the book’s opening chapter:

"There is a great deal of theory about the convergence of Markov chains. Unfortunately, none of it can be applied to get useful convergence information for most MCMC applications."

Sometimes the magic can backfire in the form of pseudo-convergence. A Markov chain can appear to have converged when in fact it has not. Again Geyer explains:

"People who have used MCMC in complicated problems can tell stories about samples that appeared to be converging until, after weeks of running, they discovered a new part of the state space and the distribution changed radically."

Despite the element of mystery in MCMC, no practical computational alternative exists for many problems. Fortunately, MCMC often works better in practice than we have reason to expect by theory. Because there is no definitive theory to settle some of the practical issues in applying MCMC, there is much room for differing preferences. The Handbook has 42 contributors and four editors; differing approaches are inevitable. For example, Andrew Gelman and Kenneth Shirley advocate monitoring convergence using multiple Markov chains in one chapter while Charles Geyer argues against this approach in another chapter.

The Handbook consists of 24 chapters, arranged into two parts, the first entitled “Foundations, Methodology, and Algorithms.” and the second “Applications and Case Studies.” The line between algorithms and applications is somewhat blurry and so some of the chapters in the first part lean more toward application than the title might lead one to expect. The first part also contains valuable historical information. This history helps identify which contemporary practices are based more on tradition than demonstrated advantage. The second part gives examples of MCMC algorithms applied to a wide variety of areas: ecology, medical imaging, social science, etc.

Handbook of Markov Chain Monte Carlo is a valuable resource for those new to MCMC as well as to experienced practitioners. It is not a handbook analogous to, say, the Boy Scout Handbook. The field of MCMC is not settled enough for such a handbook to be possible. Instead, it is a collection of valuable information regarding a powerful computational approach to evaluating complex statistical models.

Reviewed by John D. Cook for MAA, on 25th June 2011