Finite Markov Processes

A finite Markov process is a random process on a graph, where from each state you specify the probability of selecting each available transition to a new state. Finite Markov processes are used to model a variety of decision processes in areas such as games, weather, manufacturing, business, and biology.  The Wolfram Language provides complete support for both discrete-time and continuous-time finite Markov processes. The symbolic representation of a Markov process makes it easy to simulate its behavior, estimate its parameters from data, and compute state probabilities for finite and infinite time horizons, as well as find all statistical properties of the time to first hitting some target states. A full suite of structural, transient, and limiting properties for Markov processes is directly available.

Markov Process Models

DiscreteMarkovProcess represents a finite-state, discrete-time Markov process

ContinuousMarkovProcess represents a finite-state, continuous-time Markov process

HiddenMarkovProcess represents a discrete-time Markov process with emissions

Properties

MarkovProcessProperties structural, transient, and limiting properties

FirstPassageTimeDistribution distribution of times for hitting a set of states

FindHiddenMarkovStates decode hidden states from emissions

Random Process Framework »

RandomFunction simulate a discrete or continuous Markov process

EstimatedProcess estimate parameters in a discrete or continuous Markov process

StationaryDistribution limiting or conditionally stationary distribution

Discrete-Time Markov Processes »

MAProcess  ▪  ARProcess  ▪  ARMAProcess  ▪  ...

Continuous-Time Markov Processes »

WienerProcess  ▪  ItoProcess  ▪  StratonovichProcess  ▪  ...

Other Markov Processes

QueueingProcess  ▪  PoissonProcess  ▪  InhomogeneousPoissonProcess  ▪  CompoundPoissonProcess  ▪  TelegraphProcess