Finite Markov Processes
A finite Markov process is a random process on a graph, where from each state you specify the probability of selecting each available transition to a new state. Finite Markov processes are used to model a variety of decision processes in areas such as games, weather, manufacturing, business, and biology. The Wolfram Language provides complete support for both discrete-time and continuous-time finite Markov processes. The symbolic representation of a Markov process makes it easy to simulate its behavior, estimate its parameters from data, and compute state probabilities for finite and infinite time horizons, as well as find all statistical properties of the time to first hitting some target states. A full suite of structural, transient, and limiting properties for Markov processes is directly available.
Markov Process Models
DiscreteMarkovProcess — represents a finite-state, discrete-time Markov process
ContinuousMarkovProcess — represents a finite-state, continuous-time Markov process
HiddenMarkovProcess — represents a discrete-time Markov process with emissions
Properties
MarkovProcessProperties — structural, transient, and limiting properties
FirstPassageTimeDistribution — distribution of times for hitting a set of states
FindHiddenMarkovStates — decode hidden states from emissions
Random Process Framework »
RandomFunction — simulate a discrete or continuous Markov process
EstimatedProcess — estimate parameters in a discrete or continuous Markov process
StationaryDistribution — limiting or conditionally stationary distribution
Discrete-Time Markov Processes »
MAProcess ▪ ARProcess ▪ ARMAProcess ▪ ...
Continuous-Time Markov Processes »
WienerProcess ▪ ItoProcess ▪ StratonovichProcess ▪ ...
Other Markov Processes
QueueingProcess ▪ PoissonProcess ▪ InhomogeneousPoissonProcess ▪ CompoundPoissonProcess ▪ TelegraphProcess