Note that we have not covered that are certain conditions, which will be explained later. If this chain-like probability structure satisfied certain conditions, it will be called Markov Chain and is believed to be able to converge to a specific stationary state after a certain amount of sampling, regardless the initial point. Illustration of a discrete Markov Chain and its Transition Matrix This forms a chain-like structure of sampling from the three states.” Similarly, he also has a fixed pattern of breakfast choice if he ate (A) or (B) yesterday. For example, if he ate (C) Eggs yes- terday, he will have 20% chance eating (C) Eggs again and 20% chance eating (A) Bread and 70% change (B) Cereal. What he eats today is only dependent on what he ate yesterday. “Tom only has three breakfast options for his whole life. The following is a simple breakfast example of the Markov Chain: Markov Chain can be considered as a chain-like sampling method where the distribution of each sample is only dependent on the previous drawn sample. You use MCMC whenever you want to sample from the complex high dimensional distribution What is A Markov Chain Markov Chain Monte Carlo is a stochastic sampling method, normally used to sample from target distributions, which in practice are often intractable due to their complication and high dimensionality.
