Bayesian Complexity and Related Measures

 

Given a bayesian model, we introduce a novel concept called Bayesian Complexity. By taking these complexity as the payoff in a zero-sum game, the posterior distribution comes as the minimax solution of the game, hence giving  a variational version of the Bayesian paradigm. One can evaluate the "distance" between two different posteriors (i.e. computed under different prior and/or likelihoods) by looking at the difference of the expected payoff for one of the players. The connection to information theory complexity, Bregman's divergences and scoring functions is also discussed.