A finite Markov process is a random process on a graph, where from each state you specify the probability of selecting each available transition to a new state. Finite Markov processes are used to model a variety of decision processes in areas such as games, weather, manufacturing, business, and biology.
Mathematica provides complete support for both discrete-time and continuous-time finite Markov processes. The symbolic representation of a Markov process makes it easy to simulate its behavior, estimate its parameters from data, and compute state probabilities for finite and infinite time horizons, as well as find all statistical properties of the time to first hitting some target states. A full suite of structural, transient, and limiting properties for Markov processes is directly available.
DiscreteMarkovProcess — represents a finite-state discrete-time Markov process
ContinuousMarkovProcess — represents a finite-state continuous-time Markov process
MarkovProcessProperties — structural, transient, and limiting properties
FirstPassageTimeDistribution — distribution of times for hitting a set of states
RandomFunction — simulate a discrete or continuous Markov process
EstimatedProcess — estimate parameters in a discrete or continuous Markov process
StationaryDistribution — limiting or conditionally stationary distribution