Semantics of higher-order probabilistic programs with conditioning Fredrik Dahlqvist Dexter Kozen University College London Cornell University
[email protected] [email protected] Abstract—We present a denotational semantics for higher- exponentials. To accommodate conditioning and Bayesian order probabilistic programs in terms of linear operators between inference, we introduce ‘Bayesian types’, in which values are Banach spaces. Our semantics is rooted in the classical theory of decorated with a prior distribution. Based on this foundation, Banach spaces and their tensor products, but bears similarities with the well-known semantics of higher-order programs a` we give a type system and denotational semantics for an la Scott through the use ordered Banach spaces which allow idealized higher-order probabilistic language with sampling, definitions in terms of fixed points. Being based on a monoidal conditioning, and Bayesian inference. rather than cartesian closed structure, our semantics effectively We believe our approach should appeal to computer sci- treats randomness as a resource. entists, as it is true to traditional Scott-style denotational se- mantics (see IV-D). It should also appeal to mathematicians, I. INTRODUCTION statisticians and§ machine learning theorists, as it uses very Probabilistic programming has enjoyed a recent resurgence familiar mathematical objects from those fields. For example, of interest driven by new applications in machine learning a traditional perspective is that a Markov process is just a and statistical analysis of large datasets. The emergence of positive linear operator of norm 1 between certain Banach probabilistic programming languages such as Church and lattices [7, Ch. 19]. These are precisely the morphisms of our Anglican, which allow statisticians to construct and sample semantics.