ContinuousMarkovProcess

ContinuousMarkovProcess[i0,q]
represents a continuous-time finite-state Markov process with transition rate matrix q and initial state .

ContinuousMarkovProcess[p0,q]
represents a Markov process with initial state probability vector .

ContinuousMarkovProcess[,m,μ]
represents a Markov process with transition matrix m and transition rates μ.

ContinuousMarkovProcess[,g]
represents a Markov process transition rate matrix from the graph g.

DetailsDetails

  • ContinuousMarkovProcess is also known as a continuous-time Markov chain.
  • ContinuousMarkovProcess is a continuous-time and discrete-state random process.
  • The states of ContinuousMarkovProcess are integers between 1 and , where is the length of transition rate matrix q.
  • For infinitesimal time dt, gives the probability that the process transitions from state i to state j over the next dt units of time qi,jdtProbability[x[t+dt]jx[t]i].
  • The time the process stays in state i before transitioning follows ExponentialDistribution[-qii].
  • The transition matrix m specifies conditional transition probabilities mi,jProbability[x[tk+1]jx[tk]i], where is the state at time , and the transition rate specifies that the time between events in state follows ExponentialDistribution[μi].
  • EstimatedProcess[data,ContinuousMarkovProcess[n]] indicates that process with n states should be estimated.
  • The transition matrix in the case of a graph g is constructed to give equal probability of transitioning to each incident vertex with unit transition rates.
  • ContinuousMarkovProcess allows q to be an × matrix where and for with rows that sum to 0, can be an integer between 1 and , is a vector of length of non-negative elements that sum to 1, m is an × matrix with non-negative elements and rows that sum to 1, and μ is a vector of length with positive elements.
  • ContinuousMarkovProcess can be used with such functions as MarkovProcessProperties, PDF, Probability, and RandomFunction.

Background
Background

  • ContinuousMarkovProcess constructs a continuous Markov process, i.e. a continuous-time process with a finite number of states such that the probability of transitioning to a given state depends only on the current state. More precisely, processes defined by ContinuousMarkovProcess consist of states whose values come from a finite set and for which the time spent in each state has an exponential distribution. Processes defined by ContinuousMarkovProcess are sometimes referred to as continuous-time Markov chains (CTMC) and are the continuous analogues of processes constructed by DiscreteMarkovProcess, the time parameters of which are discrete.
  • Continuous Markov processes arise naturally in many areas of mathematics and physical sciences and are used to model queues, chemical reactions, electronics failures, and geological sedimentation. In addition, a considerable amount of research has gone into the understanding of continuous Markov processes from a probability theoretic perspective.
  • A number of parameter schemes may be input into ContinuousMarkovProcess. Most typically, ContinuousMarkovProcess constructs a CTMC from input parameters that specify an initial state and a transition rate matrix . The initial state may be a single integer between 1 and or a probability vector of length consisting of non-negative elements that sum to 1. The transition rate matrix is a square matrix (or SparseArray) of dimension × whose rows sum to 0 and which satisfies and for , and the states of ContinuousMarkovProcess are integers between 1 and . The time for which a given ContinuousMarkovProcess stays in state before transitioning to state is distributed according to a negative exponential distribution, i.e. tiExponentialDistribution[-qi i], where denotes the -entry of and is a shorthand for Distributed.
  • A ContinuousMarkovProcess may also be specified using a transition matrix together with associated transition rates . In this case, the entries of consist of conditional transition probabilities, i.e. the probability that given that , and the associated transition rate specifies that the time between events in state is distributed according to ExponentialDistribution[-μi]. Here, is defined to be the random time at which the ^(th) state transition has occurred, denotes the state transitioned to by the process at time , is an × matrix with non-negative elements and rows that sum to 1, and is a vector of length with positive elements.
  • Finally, ContinuousMarkovProcess can produce a CTMC from an input graph with edges denoting the transition rates between the events represented by the vertices and .
  • A number of functions can be used with ContinuousMarkovProcess, including MarkovProcessProperties, PDF, and Probability. RandomFunction can be used to simulate a CTMC, the result of which (a TemporalData object) may then be visualized via ListLinePlot, sliced via TimeSeries, and analyzed by way of functions like Mean, Variance, StandardDeviation, and Momentfunctions that can also be used to analyze distributions defined by ContinuousMarkovProcess. ContinuousMarkovProcess can also be input as the mproc parameter to FirstPassageTimeDistribution in order to analyze the distribution of times required for a CTMC to pass from its initial state to its final state for the first time, while the statistical properties of processes constructed by ContinuousMarkovProcess can be analyzed directly using functions such as CharacteristicFunction and CentralMomentGeneratingFunction.
  • In addition to their statistical properties, processes constructed by ContinuousMarkovProcess can be represented as graphs via Graph, thus inheriting a level of graph-theoretic functionality. Functions such as QueueingProcess, QueueProperties, and SurvivalFunction make use of properties of CTMC in modeling and applications. ContinuousMarkovProcess may also serve as an approximation tool by way of EstimatedProcess, a function that computes an approximate CTMC with states which best fits a given dataset.
Introduced in 2012
(9.0)
| Updated in 2014
(10.0)