DOCSLIB.ORG

Hidden Markov Models in Bioinformatics

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Hidden Markov Models in Bioinformatics Current Bioinformatics, 2007, 2, 49-61 49 Hidden Markov Models in Bioinformatics Valeria De Fonzo1, Filippo Aluffi-Pentini2 and Valerio Parisi*,3 1EuroBioPark, Università di Roma “Tor Vergata”, Via della Ricerca Scientifica 1, 00133 Roma, Italy 2Dipartimento Metodi e Modelli Matematici, Università di Roma “La Sapienza”, Via A. Scarpa 16, 00161 Roma, Italy 3Dipartimento di Medicina Sperimentale e Patologia, Università di Roma “La Sapienza”, Viale Regina Elena 324, 00161 Roma, Italy Abstract: Hidden Markov Models (HMMs) became recently important and popular among bioinformatics researchers, and many software tools are based on them. In this survey, we first consider in some detail the mathematical foundations of HMMs, we describe the most important algorithms, and provide useful comparisons, pointing out advantages and drawbacks. We then consider the major bioinformatics applications, such as alignment, labeling, and profiling of sequences, protein structure prediction, and pattern recognition. We finally provide a critical appraisal of the use and perspectives of HMMs in bioinformatics. Keywords: Hidden markov model, HMM, dynamical programming, labeling, sequence profiling, structure prediction. INTRODUCTION this case, the time evolution of the internal states can be induced only through the sequence of the observed output A Markov process is a particular case of stochastic states. process, where the state at every time belongs to a finite set, the evolution occurs in a discrete time and the probability If the number of internal states is N, the transition distribution of a state at a given time is explicitly dependent probability law is described by a matrix with N times N only on the last states and not on all the others. values; if the number of emissions is M, the emission probability law is described by a matrix with N times M A Markov chain is a first-order Markov process for values. A model is considered defined once given these two which the probability distribution of a state at a given time is matrices and the initial distribution of the internal states. explicitly dependent only on the previous state and not on all the others. In other words, the probability of the next The paper by Rabiner [1] is widely well appreciated for (“future”) state is directly dependent only on the present clarity in explaining HMMs. state and the preceding (“past”) states are irrelevant once the present state is given. More specifically there is a finite set of SOME NOTATIONS possible states, and the transitions among them are governed For the sake of simplicity, in the following notations we by a set of conditional probabilities of the next state given consider only one sequence of internal states and one the present one, called transition probabilities. The transition sequence of associated emissions, even if in some cases, as probabilities are implicitly (unless declared otherwise) we shall see later, more than one sequence is to be independent of the time and then one speaks of homo- considered. geneous, or stationary, Markov chains. Note that the inde- pendent variable along the sequence is conventionally called Here are the notations: “time” also when this is completely inappropriate; for U the set of all the N example for a DNA sequence, the “time” means the position possible internal states along the sequence. the set of all the Starting from a given initial state, the consecutive trans- X M possible external states itions from a state to the next one produce a time-evolution of the chain that is therefore completely represented by a L the length of the sequence sequence of states that a priori are to be considered random. k a time instant, where A Hidden Markov Model is a generalization of a Markov k 1,,L chain, in which each (“internal”) state is not directly observ- [] able (hence the term hidden) but produces (“emits”) an obs- sk internal state at time k , ervable random output (“external”) state, also called “emi- where s U ssion”, according to a given stationary probability law. In k a sequence of L internal S ()s1,s2,s3,…,sL states *Address correspondence to this author at the Dipartimento di Medicina Sperimentale e Patologia, Università di Roma “La Sapienza”, Viale Regina e emission at time k , where k Elena 324, 00161 Roma, Italy; Tel: +39 06 4991 0787; Fax: +39 338 ek X 09981736; E-mail: [email protected] 1574-8936/07 $50.00+.00 © 2007 Bentham Science Publishers Ltd. 50 Current Bioinformatics, 2007, Vol. 2, No. 1 De Fonzo et al. E e ,e ,e ,…,e a sequence of L external The most used algorithms are: ()1 2 3 L states 1. the forward algorithm: find the probability of emi- the probabilities of a trans- ssion distribution (given a model) starting from the au,v = Ps()k = v | sk1 = u ition to the state v from beginning of the sequence. the state u 2. the backward algorithm: find the probability of A the N N matrix of emission distribution (given a model) starting from the end of the sequence. elements au,v B) Decoding problem: given a model and a sequence of the probabilities of the bu ()x = Pe()k = x | sk = u observations, induce the most likely hidden states. emission x from the state u More specifically: 1. find the sequence of internal states that has, as a B the N M matrix of ele- whole, the highest probability. The most used ments bu ()x algorithm is the Viterbi algorithm. the probability of the initial u Ps()1 = u 2. find for each position the internal state that has the state u highest probability. The most used algorithm is the posterior decoding algorithm. the N -vector of elements u C) Learning problem: given a sequence of observations, find an optimal model. = (A,B,) the definition of the HMM model The most used algorithms start from an initial guessed model and iteratively adjust the model parameters. More A SIMPLE EXAMPLE specifically: We propose an oversimplified biological example of an 1. find the optimal model based on the most probable HMM (Fig. 1), inspired by the toy example in Eddy [2] with sequences (as in problem B1). The most used only two internal states but with exponential complexity. algorithm is the Viterbi training (that uses recursively The model is detailed in Fig. 1a. the Viterbi algorithm in B1). The set of internal states is U {}'c','n' where 'c' and 2. find the optimal model based on the sequences of 'n' stand for the coding and non-coding internal states and most probable internal states (as in problem B2). The the set of emissions is the set of the four DNA bases: most used algorithm is the Baum-Welch algorithm (that uses recursively the posterior decoding X {}' A','T','C','G' algorithm in B2). As emitted sequence, we consider a sequence of 65 bases A) THE EVALUATION PROBLEM (Fig. 1b). The probability of observing a sequence E of emissions It is important to note that in most cases of HMM use in given an HMM (likelihood function of ), is given by bioinformatics a fictitious inversion occurs between causes and effects when dealing with emissions. For example, one P(E | ) = P(E | S;) P(S | ) can synthesise a (known) polymer sequence that can have S different (unknown) features along the sequence. In an HMM one must choose as emissions the monomers of the We note that the logarithm of the likelihood function sequence, because they are the only known data, and as (log-likelihood) is more often used. internal states the features to be estimated. In this way, one The above sum must be computed over all the hypothesises that the sequence is the effect and the features N L possible sequences S (of length L ) of internal states are the cause, while obviously the reverse is true. An and therefore the direct computation is too expensive; excellent case is provided by the polypeptides, for which it is fortunately there exist some algorithms which have a just the amino acid sequence that causes the secondary considerably lower complexity, for example the forward and structures, while in an HMM the amino acids are assumed as the backward algorithms (of complexity O(N 2L), see emissions and the secondary structures are assumed as below). internal states. A1) The Forward Algorithm MAIN TYPES OF PROBLEMS This method introduces auxiliary variables k (called The main types of problems occurring in the use of forward variables), where Hidden Markov Models are: u = Pe,…,e ;s = u | is the probability of A) Evaluation problem (Direct problem): compute the k () ()1 k k observing a partial sequence of emissions e …e and a probability that a given model generates a given 1 k sequence of observations. state sk = u at time k . Hidden Markov Models in Bioinformatics Current Bioinformatics, 2007, Vol. 2, No. 1 51 Fig. (1). An example of HMM. (a) The square boxes represent the internal states 'c' (coding) and 'n' (non coding), inside the boxes there are the probabilities of each emission ( ' A', 'T', 'C' and 'G') for each state; outside the boxes four arrows are labelled with the corresponding transition probability. (b) The first row is a sample sequence of 65 observed emissions and the second row is one of the likely sequences of internal states. The boxed part is dealt with in (c) and (d). (c) The right-hand side column represents the boxed tract of bases in (b). The other columns represent, for each circled base, the two possible alternatives for the internal state ( 'c' or 'n') that emitted the base. Each row refers to the same position along the sequence. The arrows represent all possible transitions and the emissions.
Load more

Recommended publications
  • Modeling Dependence in Data: Options Pricing and Random Walks
    UNIVERSITY OF CALIFORNIA, MERCED PH.D. DISSERTATION Modeling Dependence in Data: Options Pricing and Random Walks Nitesh Kumar A dissertation submitted in partial fulfillment of the requirements for the degree Doctor of Philosophy in Applied Mathematics March, 2013 UNIVERSITY OF CALIFORNIA, MERCED Graduate Division This is to certify that I have examined a copy of a dissertation by Nitesh Kumar and found it satisfactory in all respects, and that any and all revisions required by the examining committee have been made. Faculty Advisor: Harish S. Bhat Committee Members: Arnold D. Kim Roummel F. Marcia Applied Mathematics Graduate Studies Chair: Boaz Ilan Arnold D. Kim Date Contents 1 Introduction 2 1.1 Brief Review of the Option Pricing Problem and Models . ......... 2 2 Markov Tree: Discrete Model 6 2.1 Introduction.................................... 6 2.2 Motivation...................................... 7 2.3 PastWork....................................... 8 2.4 Order Estimation: Methodology . ...... 9 2.5 OrderEstimation:Results. ..... 13 2.6 MarkovTreeModel:Theory . 14 2.6.1 NoArbitrage.................................. 17 2.6.2 Implementation Notes. 18 2.7 TreeModel:Results................................ 18 2.7.1 Comparison of Model and Market Prices. 19 2.7.2 Comparison of Volatilities. 20 2.8 Conclusion ...................................... 21 3 Markov Tree: Continuous Model 25 3.1 Introduction.................................... 25 3.2 Markov Tree Generation and Computational Tractability . ............. 26 3.2.1 Persistentrandomwalk. 27 3.2.2 Number of states in a tree of fixed depth . ..... 28 3.2.3 Markov tree probability mass function . ....... 28 3.3 Continuous Approximation of the Markov Tree . ........ 30 3.3.1 Recursion................................... 30 3.3.2 Exact solution in Fourier space .
    [Show full text]
  • Estimation of Viterbi Path in Bayesian Hidden Markov Models
    Estimation of Viterbi path in Bayesian hidden Markov models Jüri Lember1,∗ Dario Gasbarra2, Alexey Koloydenko3 and Kristi Kuljus1 May 14, 2019 1University of Tartu, Estonia; 2University of Helsinki, Finland; 3Royal Holloway, University of London, UK Abstract The article studies different methods for estimating the Viterbi path in the Bayesian framework. The Viterbi path is an estimate of the underlying state path in hidden Markov models (HMMs), which has a maximum joint posterior probability. Hence it is also called the maximum a posteriori (MAP) path. For an HMM with given parameters, the Viterbi path can be easily found with the Viterbi algorithm. In the Bayesian framework the Viterbi algorithm is not applicable and several iterative methods can be used instead. We introduce a new EM-type algorithm for finding the MAP path and compare it with various other methods for finding the MAP path, including the variational Bayes approach and MCMC methods. Examples with simulated data are used to compare the performance of the methods. The main focus is on non-stochastic iterative methods and our results show that the best of those methods work as well or better than the best MCMC methods. Our results demonstrate that when the primary goal is segmentation, then it is more reasonable to perform segmentation directly by considering the transition and emission parameters as nuisance parameters. Keywords: HMM, Bayes inference, MAP path, Viterbi algorithm, segmentation, EM, variational Bayes, simulated annealing 1 Introduction and preliminaries arXiv:1802.01630v2 [stat.CO] 11 May 2019 Hidden Markov models (HMMs) are widely used in application areas including speech recognition, com- putational linguistics, computational molecular biology, and many more.
    [Show full text]
  • A Study of Hidden Markov Model
    University of Tennessee, Knoxville TRACE: Tennessee Research and Creative Exchange Masters Theses Graduate School 8-2004 A Study of Hidden Markov Model Yang Liu University of Tennessee - Knoxville Follow this and additional works at: https://trace.tennessee.edu/utk_gradthes Part of the Mathematics Commons Recommended Citation Liu, Yang, "A Study of Hidden Markov Model. " Master's Thesis, University of Tennessee, 2004. https://trace.tennessee.edu/utk_gradthes/2326 This Thesis is brought to you for free and open access by the Graduate School at TRACE: Tennessee Research and Creative Exchange. It has been accepted for inclusion in Masters Theses by an authorized administrator of TRACE: Tennessee Research and Creative Exchange. For more information, please contact [email protected]. To the Graduate Council: I am submitting herewith a thesis written by Yang Liu entitled "A Study of Hidden Markov Model." I have examined the final electronic copy of this thesis for form and content and recommend that it be accepted in partial fulfillment of the equirr ements for the degree of Master of Science, with a major in Mathematics. Jan Rosinski, Major Professor We have read this thesis and recommend its acceptance: Xia Chen, Balram Rajput Accepted for the Council: Carolyn R. Hodges Vice Provost and Dean of the Graduate School (Original signatures are on file with official studentecor r ds.) To the Graduate Council: I am submitting herewith a thesis written by Yang Liu entitled “A Study of Hidden Markov Model.” I have examined the final electronic copy of this thesis for form and content and recommend that it be accepted in partial fulfillment of the requirements for the degree of Master of Science, with a major in Mathematics.
    [Show full text]
  • Sequence Models
    Sequence Models Noah Smith Computer Science & Engineering University of Washington [email protected] Lecture Outline 1. Markov models 2. Hidden Markov models 3. Viterbi algorithm 4. Other inference algorithms for HMMs 5. Learning algorithms for HMMs Shameless Self-Promotion • Linguistic Structure Prediction (2011) • Links material in this lecture to many related ideas, including some in other LxMLS lectures. • Available in electronic and print form. MARKOV MODELS One View of Text • Sequence of symbols (bytes, letters, characters, morphemes, words, …) – Let Σ denote the set of symbols. • Lots of possible sequences. (Σ* is infinitely large.) • Probability distributions over Σ*? Pop QuiZ • Am I wearing a generative or discriminative hat right now? Pop QuiZ • Generative models tell a • Discriminative models mythical story to focus on tasks (like explain the data. sorting examples). Trivial Distributions over Σ* • Give probability 0 to sequences with length greater than B; uniform over the rest. • Use data: with N examples, give probability N-1 to each observed sequence, 0 to the rest. • What if we want every sequence to get some probability? – Need a probabilistic model family and algorithms for constructing the model from data. A History-Based Model n+1 p(start,w1,w2,...,wn, stop) = γ(wi w1,w2,...,wi 1) | − i=1 • Generate each word from left to right, conditioned on what came before it. Die / Dice one die two dice start one die per history: … … … start I one die per history: … … … history = start start I want one die per history: … … history = start I … start I want a one die per history: … … … history = start I want start I want a flight one die per history: … … … history = start I want a start I want a flight to one die per history: … … … history = start I want a flight start I want a flight to Lisbon one die per history: … … … history = start I want a flight to start I want a flight to Lisbon .
    [Show full text]
  • Entropy Rate
    Lecture 6: Entropy Rate • Entropy rate H(X) • Random walk on graph Dr. Yao Xie, ECE587, Information Theory, Duke University Coin tossing versus poker • Toss a fair coin and see and sequence Head, Tail, Tail, Head ··· −nH(X) (x1; x2;:::; xn) ≈ 2 • Play card games with friend and see a sequence A | K r Q q J ♠ 10 | ··· (x1; x2;:::; xn) ≈? Dr. Yao Xie, ECE587, Information Theory, Duke University 1 How to model dependence: Markov chain • A stochastic process X1; X2; ··· – State fX1;:::; Xng, each state Xi 2 X – Next step only depends on the previous state p(xn+1jxn;:::; x1) = p(xn+1jxn): – Transition probability pi; j : the transition probability of i ! j P = j – p(xn+1) xn p(xn)p(xn+1 xn) – p(x1; x2; ··· ; xn) = p(x1)p(x2jx1) ··· p(xnjxn−1) Dr. Yao Xie, ECE587, Information Theory, Duke University 2 Hidden Markov model (HMM) • Used extensively in speech recognition, handwriting recognition, machine learning. • Markov process X1; X2;:::; Xn, unobservable • Observe a random process Y1; Y2;:::; Yn, such that Yi ∼ p(yijxi) • We can build a probability model Yn−1 Yn n n p(x ; y ) = p(x1) p(xi+1jxi) p(yijxi) i=1 i=1 Dr. Yao Xie, ECE587, Information Theory, Duke University 3 Time invariance Markov chain • A Markov chain is time invariant if the conditional probability p(xnjxn−1) does not depend on n p(Xn+1 = bjXn = a) = p(X2 = bjX1 = a); for all a; b 2 X • For this kind of Markov chain, define transition matrix 2 3 6 ··· 7 6P11 P1n7 = 6 ··· 7 P 46 57 Pn1 ··· Pnn Dr.
    [Show full text]
  • Markov Decision Process Example
    Markov Decision Process Example Clarifying Stanleigh vernalised consolingly while Matthus always converts his lev lynch belligerently, he crossbreeding so subcutaneously. andDemetris air-minded. missend Answerable his heisters and touch-types pilose Tuckie excessively flanged heror hortatively flagstaffs turfafter while Hannibal Orbadiah effulging scrap and some emendated augmenters ungraciously, symptomatically. elaborative The example when an occupancy grid example of a crucial role of dots stimulus consists of theories, if π is. Below illustrates the example based on the next section. Data structure to read? MDP Illustrated mdp schematic MDP Example S 11 12 13 21 23 31 32 33 41 42 43 A. Both models by comparison of teaching mdp example, a trial starts after responding, and examples published in this in this technique for finding approximate solutions. A simple GUI and algorithms to experiment with Markov Decision Process miromanninomarkov-decision-process-examples. Markov Decision Processes Princeton University Computer. Senior ai scientist passionate about getting it simply runs into the total catch games is a moving object and examples are. Markov decision processes c Vikram Krishnamurthy 2013 6 2 Application Examples 21 Finite state Markov Decision Processes MDP xk is a S state Markov. Real-life examples of Markov Decision Processes Cross. Due to solve in a system does not equalizing strategy? First condition this remarkable and continues to. Example achieving a state satisfying property P at minimal. Example 37 Recycling Robot MDP The recycling robot Example 33 can be turned into his simple example had an MDP by simplifying it and providing some more. How does Markov decision process work? In state values here are you can be modeled as if we first accumulator hits its solutions are.
    [Show full text]
  • 1. Markov Models
    1. Markov models 1.1 Markov-chain Let X be a random variable X = (X1,...,Xt) taking values in some set S = {s1, . , sN }. The sequence is Markov chain if it has the following properties: 1. Limited horizon: P (Xt+1 = sk|X1,...,Xt) = P (Xt+1 = sk|Xt) (1) 2. Time invariant (stationary): P (X2 = sk|X1 = sj) = P (Xt+1 = sk|Xt = sj), ∀t, k, j (2) The subsequent variables may be dependent. In the general (non-Markovian) case, a variable Xt may depend on the all previous variables X1,...,Xt−1. Transition matrix A Markov model can be represented as a set of states (observations) S, and a transition matrix A. For example, S = {a, e, i, h, t, p} and aij = P (Xt+1 = sj|Xt = si): ai,j a e i h t p a 0.6 0 0 0 0 0.4 e ...... i 0.6 0.4 h ...... t 0.3 0 0.3 0.4 0 0 p ...... s0 0.9 0.1 Initial state probabilitites πsi can be represented as transitions from an auxil- liary initial state s0. 2 Markov model as finite-state automaton • Finite-State Automaton (FSA): states and transitions. • Weighted or probabilistic automaton: each transition has a probability, and transitions leaving a state sum to one. • A Markov model can be represented as a FSA. Observations either in states or transitions. b 0.6 1.0 0.9 b / 0.6 c / 1.0 c / 0.9 0.4 c / 0.4 a c 0.1 a / 0.1 3 Probability of a sequence: Given a Markov model (set of states S and transition matrix A) and a sequence of states (observations) X, it is straight- forward to compute the probability of the sequence: P (X1 ...XT ) = P (X1)P (X2|X1)P (X3|X1X2) ...P (XT |X1 ...XT −(3)1) = P (X1)P (X2|X1)P (X3|X2) ...P (XT |XT −1) (4) T −1 Y = πX1 aXtXt+1 (5) t=1 where aij is the transition probability from state i to state j, and πX1 is the initial state probability.
    [Show full text]
  • Viterbi Extraction Tutorial with Hidden Markov Toolkit
    Viterbi Extraction tutorial with Hidden Markov Toolkit Zulkarnaen Hatala 1, Victor Puturuhu 2 Electrical Engineering Department Politeknik Negeri Ambon Ambon, Indonesia [email protected] [email protected] Abstract —An algorithm used to extract HMM parameters condition probabilities. The distribution of observation only is revisited. Most parts of the extraction process are taken from depends on the current state of Markov chain. In some implemented Hidden Markov Toolkit (HTK) program under application not all states of Markov chain emit observations. name HInit. The algorithm itself shows a few variations Set of states that emit an observation is called emitting states, compared to another domain of implementations. The HMM while non emitting states could be an entry or start state, model is introduced briefly based on the theory of Discrete absorption or exiting state and intermediate or delaying state. Time Markov Chain. We schematically outline the Viterbi The entry state is the Markov chain state with initial method implemented in HTK. Iterative definition of the probability 1. method which is ready to be implemented in computer programs is reviewed. We also illustrate the method One application of Hidden Markov Model is in speech calculation precisely using manual calculation and extensive recognition. The observations of HMM are used to model graphical illustration. The distribution of observation sequence of speech feature vectors like Mel Frequency probability used is simply independent Gaussians r.v.s. The Cepstral Coefficients (MFCC) [4]. MFCC is assumed to be purpose of the content is not to justify the performance or generated by emitting hidden states of the underlying accuracy of the method applied in a specific area.
    [Show full text]
  • Markov Chains and Hidden Markov Models
    COMP 182 Algorithmic Thinking Luay Nakhleh Markov Chains and Computer Science Hidden Markov Models Rice University ❖ What is p(01110000)? ❖ Assume: ❖ 8 independent Bernoulli trials with success probability of α? ❖ Answer: ❖ (1-α)5α3 ❖ However, what if the assumption of independence doesn’t hold? ❖ That is, what if the outcome in a Bernoulli trial depends on the outcomes of the trials the preceded it? Markov Chains ❖ Given a sequence of observations X1,X2,…,XT ❖ The basic idea behind a Markov chain (or, Markov model) is to assume that Xt captures all the relevant information for predicting the future. ❖ In this case: T p(X1X2 ...XT )=p(X1)p(X2 X1)p(X3 X2) p(XT XT 1)=p(X1) p(Xt Xt 1) | | ··· | − | − t=2 Y Markov Chains ❖ When Xt is discrete, so Xt∈{1,…,K}, the conditional distribution p(Xt|Xt-1) can be written as a K×K matrix, known as the transition matrix A, where Aij=p(Xt=j|Xt-1=i) is the probability of going from state i to state j. ❖ Each row of the matrix sums to one, so this is called a stochastic matrix. 590 Chapter 17. Markov and hidden Markov models Markov Chains 1 α 1 β − α − A11 A22 A33 1 2 A12 A23 1 2 3 β ❖ A finite-state Markov chain is equivalent(a) (b) Figure 17.1 State transition diagrams for some simple Markov chains. Left: a 2-state chain. Right: a to a stochastic automaton3-state left-to-right. chain. ❖ One way to represent aAstationary,finite-stateMarkovchainisequivalenttoa Markov chain is stochastic automaton.Itiscommon 590 to visualizeChapter such 17.
    [Show full text]
  • N-Best Hypotheses
    FIANAL PROJECT REPORT NAME: Shuan wang STUDENT #: 21224043 N-best Hypotheses [INDEX] 1. Introduction 1.1 Project Goal 1.2 Sources of Interest 1.3 Current Work 2. Implemented Three Algorithms 2.1 General Viterbi Algorithm 2.1.1 Basic Ideas 2.1.2 Implementation 2.2 N-Best List in HMM 2.2.1 Basic Ideas 2.2.2 Implementation 2.3 BMMF (Best Max-Marginal First) 2.3.1 Basic Ideas 2.3.2 Implementation 3. Some Experiment Results 4. Future Work 5. References 1 of 8 FIANAL PROJECT REPORT NAME: Shuan wang STUDENT #: 21224043 1. Introduction It’s very useful to find the top N most probable figures of hidden variable sequence. There are some important methods in recent years to find such kind of N figures: (1) General Viterbi Algorithm (2) MFP (Max-Flow Propagation), by Nilson and Lauritzen in 1998 (3) N-Best List in HMM, by Nilson and Goldberger in 2001 (4) BMMF (Best Max-Marginal First), by Chen Yanover and Yair Weiss in 2003 In this project my task is to realize part of these algorithms and test the results. 1.1 Project Goal The project goal is to totally understand the basic algorithm ideas and try to realize the most recent three of them (MFP, N-Best List in HMM, BMMF) and test them. 1.2 Sources of Interest Many applications, such as protein folding, vocabulary speech recognition and image analysis, want to find not just the best configuration of the hidden variables (state sequence), but rather the top N. Normally the recognizers in these applications are based on a relatively simple model.
    [Show full text]
  • Notes on Markov Models for 16.410 and 16.413 1 Markov Chains
    Notes on Markov Models for 16.410 and 16.413 1 Markov Chains A Markov chain is described by the following: • a set of states S = fs1; s2; : : : sng • a set of transition probabilities T (si; sj ) = p(sj jsi) • an initial state s0 2 S The Markov Assumption The state at time t, st, depends only on the previous state st−1 and not the previous history. That is, p(stjst−1; st−2; st−3; s0) = p(stjst−1) (1) Things you might want to know about Markov chains: • Probability of being in state si at time t • Stationary distribution 2 Markov Decision Processes The extension of Markov chains to decision making A Markov decision process (MDP) is a model for deciding how to act in “an accessible, stochastic environment with a known transition model” (Russell & Norvig, pg 500.). A Markov decision process is described by the following: • a set of states S = fs1; s2; : : : sng • a set of actions A = fa1; a2; : : : ; amg • a set of transition probabilities T (si; a; sj ) = p(sjjsi; a) • a set of rewards R : S × A 7! < • a discount factor γ 2 [0; 1] • an initial state s0 2 S Things you might want to know about MDPs: • The optimal policy One way to compute the optimal policy: Define the optimal value function V (si) by the Bellman equation: jSj V (si) = max 0R(si; a) + γ p(sj jsi; a) · V (sj )1 (2) a Xj=1 @ A The value iteration algorithm using Bellman’s equation: 1 1. t = 0 0 2.
    [Show full text]
  • A HMM Approach to Identifying Distinct DNA Methylation Patterns
    A HMM Approach to Identifying Distinct DNA Methylation Patterns for Subtypes of Breast Cancers Thesis Presented in Partial Fulfillment of the Requirements for the Degree Master of Science in the Graduate School of The Ohio State University By Maoxiong Xu, B.S. Graduate Program in Computer Science and Engineering The Ohio State University 2011 Thesis Committee: Victor X. Jin, Advisor Raghu Machiraju Copyright by Maoxiong Xu 2011 Abstract The United States has the highest annual incidence rates of breast cancer in the world; 128.6 per 100,000 in whites and 112.6 per 100,000 among African Americans.[1,2] It is the second-most common cancer (after skin cancer) and the second-most common cause of cancer death (after lung cancer).[1] Recent studies have demonstrated that hyper- methylation of CpG islands may be implicated in tumor genesis, acting as a mechanism to inactivate specific gene expression of a diverse array of genes (Baylin et al., 2001). Genes have been reported to be regulated by CpG hyper-methylation, include tumor suppressor genes, cell cycle related genes, DNA mismatch repair genes, hormone receptors and tissue or cell adhesion molecules (Yan et al., 2001). Usually, breast cancer cells may or may not have three important receptors: estrogen receptor (ER), progesterone receptor (PR), and HER2. So we will consider the ER, PR and HER2 while dealing with the data. In this thesis, we first use Hidden Markov Model (HMM) to train the methylation data from both breast cancer cells and other cancer cells. Also we did hierarchy clustering to the gene expression data for the breast cancer cells and based on the clustering results, we get the methylation distribution in each cluster.
    [Show full text]