DiscreteMarkovProcess

DiscreteMarkovProcess[i0,m]
represents a discrete-time, finite-state Markov process with transition matrix m and initial state i0.

DiscreteMarkovProcess[p0,m]
represents a Markov process with initial state probability vector p0.

DiscreteMarkovProcess[,g]
represents a Markov process with transition matrix from the graph g.

DetailsDetails

  • DiscreteMarkovProcess is also known as a discrete-time Markov chain.
  • DiscreteMarkovProcess is a discrete-time and discrete-state random process.
  • The states of DiscreteMarkovProcess are integers between 1 and , where is the length of transition matrix m.
  • The transition matrix m specifies conditional transition probabilities mi,jProbability[x[k+1]jx[k]i], where x[k] is the state of the process at time k. »
  • A discrete Markov process can be seen as a random walk on a graph, where the probability of transitioning from state i to state j is specified by mi,j.
  • EstimatedProcess[data,DiscreteMarkovProcess[n]] indicates that a process with n states should be estimated.
  • The transition matrix in the case of a graph g is constructed to give equal probability of transitioning to each incident vertex.
  • DiscreteMarkovProcess allows m to be an × matrix with non-negative elements and rows that sum to 1, i0 is an integer between 1 and , and p0 is a vector of length of non-negative elements that sum to 1.
  • DiscreteMarkovProcess can be used with such functions as MarkovProcessProperties, PDF, Probability, and RandomFunction.

ExamplesExamplesopen allclose all

Basic Examples  (2)Basic Examples  (2)

Define a discrete Markov process:

In[1]:=
Click for copyable input

Simulate it:

In[2]:=
Click for copyable input
Out[2]=
In[3]:=
Click for copyable input
Out[3]=

Find the PDF for the state at time :

In[1]:=
Click for copyable input
In[2]:=
Click for copyable input
Out[2]=

Find the long-run proportion of time the process is in state 2:

In[3]:=
Click for copyable input
Out[3]=
Introduced in 2012
(9.0)
| Updated in 2014
(10.0)