Modeling with Bounded Partition Functions

Modeling with BoundedPartition FunctionsRyan Prescott AdamsCavendish LaboratoryUniversity of Cambridgehttp://www.inference.phy.cam.ac.uk/rpa23/16 July 2008Overall Talk MessageI Energy functions are nice models for data.I Inference in energy models is often hard.I If you can draw exact samples, you can doMCMC inference.I I have a trick for generating exact data frommany energy models.I This trick is probably a bad idea.OutlineMotivationExamples of Energy ModelsInferenceQuick Review of MCMCDoubly-Intractable Posterior DistributionsExchange SamplingConceptAuxiliary VariablesBaby and ToyExact Sampling from Energy ModelsOutlineMotivationExamples of Energy ModelsInferenceQuick Review of MCMCDoubly-Intractable Posterior DistributionsExchange SamplingConceptAuxiliary VariablesBaby and ToyExact Sampling from Energy ModelsEnergy-based Models of DataFor some spaceX , write an energy: E(x; )Turn this into a probability distribution via:1p(xj) = expf E(x; )gZ()Big energy implies small probability.ZNormalised byZ() = dx expf E(x; )g:XI Called the partition function.I Depends on the parameters.I Intractable in many interesting models.Examples of Energy-based ModelsExponential Family DistributionsTE(x; ) = T(x)+h(x)I Gaussian, Gamma, Poisson, etc.I Typically easy.Examples of Energy-based ModelsUndirected Graphical ModelsT TT T TE(x; ) = x Vx h Hh x Jh x h I Ising/Potts models, Boltzmann machinesI Perhaps hidden ...