represents a continuous-time finite-state Markov process with transition rate matrix q and initial state i0.
represents a Markov process with initial state probability vector p0.
represents a Markov process with transition matrix m and transition rates μ.
represents a Markov process transition rate matrix from the graph g.
- ContinuousMarkovProcess is also known as a continuous-time Markov chain.
- ContinuousMarkovProcess is a continuous-time and discrete-state random process.
- The states of ContinuousMarkovProcess are integers between 1 and , where is the length of transition rate matrix q.
- For infinitesimal time dt, q〚i,j〛dt gives the probability that the process transitions from state i to state j over the next dt units of time q〚i,j〛dtProbability[x[t+dt]jx[t]i].
- The time the process stays in state i before transitioning follows ExponentialDistribution[-qii].
- The transition matrix m specifies conditional transition probabilities m〚i,j〛Probability[x[tk+1]jx[tk]i], where x[tk] is the state at time tk, and the transition rate μi specifies that the time between events in state follows ExponentialDistribution[μi].
- EstimatedProcess[data,ContinuousMarkovProcess[n]] indicates that process with n states should be estimated.
- The transition matrix in the case of a graph g is constructed to give equal probability of transitioning to each incident vertex with unit transition rates.
- ContinuousMarkovProcess allows q to be an × matrix where and for with rows that sum to 0, i0 can be an integer between 1 and , p0 is a vector of length of non-negative elements that sum to 1, m is an × matrix with non-negative elements and rows that sum to 1, and μ is a vector of length with positive elements.
- ContinuousMarkovProcess can be used with such functions as MarkovProcessProperties, PDF, Probability, and RandomFunction.
Background & Context
- ContinuousMarkovProcess constructs a continuous Markov process, i.e. a continuous-time process with a finite number of states such that the probability of transitioning to a given state depends only on the current state. More precisely, processes defined by ContinuousMarkovProcess consist of states whose values come from a finite set and for which the time spent in each state has an exponential distribution. Processes defined by ContinuousMarkovProcess are sometimes referred to as continuous-time Markov chains (CTMC) and are the continuous analogues of processes constructed by DiscreteMarkovProcess, the time parameters of which are discrete.
- Continuous Markov processes arise naturally in many areas of mathematics and physical sciences and are used to model queues, chemical reactions, electronics failures, and geological sedimentation. In addition, a considerable amount of research has gone into the understanding of continuous Markov processes from a probability theoretic perspective.
- A number of parameter schemes may be input into ContinuousMarkovProcess. Most typically, ContinuousMarkovProcess constructs a CTMC from input parameters that specify an initial state and a transition rate matrix . The initial state may be a single integer between 1 and or a probability vector of length consisting of non-negative elements that sum to 1. The transition rate matrix is a square matrix (or SparseArray) of dimension × whose rows sum to 0 and which satisfies and for , and the states of ContinuousMarkovProcess are integers between 1 and . The time for which a given ContinuousMarkovProcess stays in state before transitioning to state is distributed according to an exponential distribution, i.e. tiExponentialDistribution[-qi i], where denotes the -entry of and is a shorthand for Distributed.
- A ContinuousMarkovProcess may also be specified using a transition matrix together with associated transition rates . In this case, the entries of consist of conditional transition probabilities, i.e. the probability that given that , and the associated transition rate specifies that the time between events in state is distributed according to ExponentialDistribution[μi]. Here, is defined to be the random time at which the state transition has occurred, denotes the state transitioned to by the process at time , is an × matrix with non-negative elements and rows that sum to 1, and is a vector of length with positive elements.
- Finally, ContinuousMarkovProcess can produce a CTMC from an input graph with edges denoting the transition rates between the events represented by the vertices and .
- A number of functions can be used with ContinuousMarkovProcess, including MarkovProcessProperties, PDF, and Probability. RandomFunction can be used to simulate a CTMC, the result of which (a TemporalData object) may then be visualized via ListLinePlot, sliced via TimeSeries, and analyzed by way of functions like Mean, Variance, StandardDeviation, and Moment—functions that can also be used to analyze distributions defined by ContinuousMarkovProcess. ContinuousMarkovProcess can also be input as the mproc parameter to FirstPassageTimeDistribution in order to analyze the distribution of times required for a CTMC to pass from its initial state to its final state for the first time, while the statistical properties of processes constructed by ContinuousMarkovProcess can be analyzed directly using functions such as CharacteristicFunction and CentralMomentGeneratingFunction.
- In addition to their statistical properties, processes constructed by ContinuousMarkovProcess can be represented as graphs via Graph, thus inheriting a level of graph-theoretic functionality. Functions such as QueueingProcess, QueueProperties, and SurvivalFunction make use of properties of CTMC in modeling and applications. ContinuousMarkovProcess may also serve as an approximation tool by way of EstimatedProcess, a function that computes an approximate CTMC with states which best fits a given dataset.
Examplesopen allclose all
Basic Examples (2)
Generalizations & Extensions (2)
A hospital owns two identical and independent power generators. The time to breakdown for each is exponential with parameter , and the time for repair of a malfunctioning generator is exponential with parameter . Assuming both generators were in order at , find the probability that both generators remain functional at time . The number of working generators can be modeled with a continuous-time Markov chain, with the state number representing 1 plus the number of working generators:
A flammable product is stored in a special tank at a filling station. Customers asking for the product arrive according to a Poisson process with rate . Each customer asks for one unit of the product. Any demand that occurs when the tank is out of stock is lost. Opportunities to replenish the stock in the tank occur according to a Poisson process with rate . The two Poisson processes are assumed to be independent of each other. For reasons of security, the stock can only be replenished when the tank is empty. At those opportunities, the stock is replenished with units:
A router receives packets from a group of users and transmits them over a single transmission line. Suppose packets arrive according to a Poisson process at a rate of one packet every 4 milliseconds and packet transmission times are exponentially distributed with mean 3 milliseconds. Approximate an infinite router capacity:
The number of flaws appearing on a polished mirror surface is a Poisson random variable. For a mirror with an area of , the probability of no flaws is 0.91. Find the probability of no flaws on the surface of another mirror with an area of , fabricated using the same process:
The arrival pattern of cars to an oil change center follows a Poisson process at the rate of four per hour. There is a single available mechanic, and the time taken to perform an oil change is exponentially distributed and requires an average of 12 minutes to carry out. Approximate an unlimited car space:
Vehicles in a certain country are required to be assessed every year for roadworthiness. At one vehicle assessment center, drivers wait for an average of 15 minutes before the roadworthiness assessment of their vehicle commences. The assessment takes on average 20 minutes to complete. Following the assessment, 80% of vehicles are passed as roadworthy, allowing the driver to drive home. A further 15% of vehicles are categorized as a "minor fail"; these vehicles require on average 30 minutes of repair work before the driver is allowed to drive home. The remaining 5% of vehicles are categorized as a "significant fail"; these vehicles require on average three hours of repair work before the driver can go home. Model the operation of the vehicle assessment center using a continuous-time Markov model, with states W (waiting for assessment), A (assessment taking place), M (minor repair taking place), S (significant repair taking place), and H (traveling home):
Consider a system with exponential interarrival and service times. There is one server and room for two customers: the one in service and another that is waiting. Let the number of customers in the room at time be . This is an M/M/1/2 queue; model it as a birth-death process with the following transition rate matrix, where the states are indexed as :
This agrees with the stationary distribution for QueueingProcess:
This agrees with the result from QueueProperties:
The Hubble space telescope carries six gyroscopes, with a minimum of three required for full accuracy. The operating times of the gyroscopes are independent and exponentially distributed with failure rate . If a fourth gyroscope fails, the telescope goes into sleep mode, in which further observations are suspended. It requires an exponential time with mean 1/ to put the telescope into sleep mode, after which the base station on Earth receives a sleep signal, and a shuttle mission is prepared. It takes an exponential time with mean 1/ before the repair crew arrives at the telescope and repairs the stabilizing unit with the gyroscopes. In the meantime, the other two gyroscopes may fail. If the last gyroscope fails, the telescope will crash. Suppose , , and , all with units of inverse years: