Finite Markov Processes

A finite Markov process is a random process on a graph, where from each state you specify the probability of selecting each available transition to a new state. Finite Markov processes are used to model a variety of decision processes in areas such as games, weather, manufacturing, business, and biology.

Mathematica provides complete support for both discrete-time and continuous-time finite Markov processes. The symbolic representation of a Markov process makes it easy to simulate its behavior, estimate its parameters from data, and compute state probabilities for finite and infinite time horizons, as well as find all statistical properties of the time to first hitting some target states. A full suite of structural, transient, and limiting properties for Markov processes is directly available.