hmm-tutorial

Hidden Markov ModelsPhil Blunsom pcbl@cs.mu.oz.auAugust 19, 2004AbstractThe Hidden Markov Model (HMM) is a popular statistical tool for modelling a widerange of time series data. In the context of natural language processing(NLP), HMMs havebeen applied with great success to problems such as part-of-speech tagging and noun-phrasechunking.1 IntroductionThe Hidden Markov Model(HMM) is a powerful statistical tool for modeling generative se-quencesthatcanbecharacterisedbyanunderlyingprocessgeneratinganobservablesequence.HMMs have found application in many areas interested in signal processing, and in particularspeech processing, but have also been applied with success to low level NLP tasks such aspart-of-speech tagging, phrase chunking, and extracting target information from documents.Andrei Markov gave his name to the mathematical theory of Markov processes in the earlytwentieth century[3], but it was Baum and his colleagues that developed the theory of HMMsin the 1960s[2].Markov Processes Diagram 1 depicts an example of a Markov process. The modelpresenteddescribesasimplemodelforastockmarketindex. Themodelhasthreestates,Bull,Bear and Even, and three index observations up, down, unchanged. The model is a nite stateautomaton, with probabilistic transitions between states. Given a sequence of observations,example: up-down-down we can easily verify that the state sequence that produced thoseobservations was: Bull-Bear-Bear, and the probability of the seq ...