Finite Markov Processes

A finite Markov process is a random process on a graph, where from each state you specify the probability of selecting each available transition to a new state. Finite Markov processes are used to model a variety of decision processes in areas such as games, weather, manufacturing, business, and biology.

Mathematica provides complete support for both discrete-time and continuous-time finite Markov processes. The symbolic representation of a Markov process makes it easy to simulate its behavior, estimate its parameters from data, and compute state probabilities for finite and infinite time horizons, as well as find all statistical properties of the time to first hitting some target states. A full suite of structural, transient, and limiting properties for Markov processes is directly available.

ReferenceReference

Markov Process Models

DiscreteMarkovProcess represents a finite-state discrete-time Markov process

ContinuousMarkovProcess represents a finite-state continuous-time Markov process

Properties

MarkovProcessProperties structural, transient, and limiting properties

FirstPassageTimeDistribution distribution of times for hitting a set of states

Random Process Framework »

RandomFunction simulate a discrete or continuous Markov process

EstimatedProcess estimate parameters in a discrete or continuous Markov process

StationaryDistribution limiting or conditionally stationary distribution

Related Processes

QueueingProcess ▪ PoissonProcess ▪ TelegraphProcess

Discrete-Time Markov Processes »

MAProcess ▪ ARProcess ▪ ARMAProces ▪ ...

Continuous-Time Markov Processes »

WienerProcess ▪ ItoProcess ▪ StratonovichProcess ▪ ...

New to Mathematica? Find your learning path »
Have a question? Ask support »