DiscreteMarkovProcess[i0, m]
represents a discrete-time, finite-state Markov process with transition matrix m and initial state .

DiscreteMarkovProcess[p0, m]
represents a Markov process with initial state probability vector .

DiscreteMarkovProcess[..., g]
represents a Markov process with transition matrix from the graph g.


  • DiscreteMarkovProcess is also known as a discrete-time Markov chain.
  • DiscreteMarkovProcess is a discrete-time and discrete-state random process.
  • The states of DiscreteMarkovProcess are integers between 1 and , where is the length of transition matrix m.
  • The transition matrix m specifies conditional transition probabilities mi, j=Probability[x[k+1]=jConditionedx[k]=i], where is the state of the process at time k. »
  • A discrete Markov process can be seen as a random walk on a graph, where the probability of transitioning from state to state is specified by .
  • The transition matrix in the case of a graph g is constructed to give equal probability of transitioning to each incident vertex.
  • DiscreteMarkovProcess allows m to be an × matrix with non-negative elements and rows that sum to 1, is an integer between 1 and , and is a vector of length of non-negative elements that sum to 1.
  • DiscreteMarkovProcess can be used with such functions as MarkovProcessProperties, PDF, Probability, and RandomFunction.
New in 9
New to Mathematica? Find your learning path »
Have a question? Ask support »