Markov Chains

A Markov chain is a sequence of random values whose probabilities at a time interval depends upon the value of the number at the previous time. A simple example is the nonreturning random walk, where the walkers are restricted to not go back to the location just previously visited.

Algorithmic Images (Perfectly Random Sampling with Markov Chains, Gallery, American Mathematical Society, 2002)

(Java Applets, Frank Wattenberg, Dep. Of Mathematics, Montana State University, 1997) (Digital Music Programming, Midi)


Non-Scientific Papers
(Markov Chain in Gambling, Game Theory, Lotto and Lottery, Ion Saliu, 2003) (War and Markov Chain)

Scientific Papers
(Markov Chains and Stochastic Stability, Sean Meyn, Richard Tweedie, 1994)
(Markov Chains are used to model processes such as behavior of queuing networks, Dianne OÇLeary, 2001)

Applications in
Computer Science (Probabilistic Inference using Markov Chain Monte Carlo Methods, Radford M. Neal, Dept. of Computer Science, University of Toronto, 1993)

Applications in Economy (The Evolution of Tax Evasion in the Czech Republic: A Markov Chain Analysis, Jan Hanousek ( and Filip Palda (, Public Economics from Economics Working Paper Archive at WUSTL, 2003) (Modeling Custopmer Relationships as Markov Chains, Phillip E. Pfeifer and Robert L. Carraway, Darden School of Business, Charlotteville, VA, Journal of Interactive Marketing, 2000) (Analysing economic Growth using Panel Data and Markov Chains, Guido Pellegrini, Dep. Of Statistics, University of Bologna)

Applications in Geology 

Applications in Medicine (Introduction to continuous and discrete Markov chains, including the "birth and death" process, The Connexion Project, Rice University)

Applications in Music (Modeling Music as Markov Chains - Composer Identification, Yi-Wen Liu, Music 254 Final Report, Stanford University, 2002) (Markov Chains as Tool for Jazz Improvisation Analysis, David M. Franz, Faculty of the Virginia Polytechnic, Institute and State University, Blackburg, Virginia)