Delay Differential Equations

A delay differential equation is a differential equation where the time derivatives at the current time depend on the solution and possibly its derivatives at previous times:

Instead of a simple initial condition, an initial history function needs to be specified. The quantities and are called the delays or time lags. The delays may be constants, functions and of (time-dependent delays), or functions and (state-dependent delays). Delay equations with delays of the derivatives are referred to as neutral delay differential equations (NDDEs).

The equation processing code in NDSolve has been designed so that you can input a delay differential equation in essentially mathematical notation.

x[t-τ]dependent variable with delay
x[t /; tt0]ϕspecification of initial history function as expression for less than

Inputting delays and initial history.

Currently, the implementation for DDEs in NDSolve only supports constant delays.

Solve a second-order delay differential equation.
Click for copyable input
Plot the solution and its first two derivatives.
Click for copyable input

For simplicity, this documentation is written assuming that integration always proceeds from smaller to larger . However, NDSolve supports integration in the other direction if the initial history function is given for values above and the delays are negative.

Solve a second-order delay differential equation in the direction of negative .
Click for copyable input

Comparison and Contrast with ODEs

While DDEs look a lot like ODEs the theory for them is quite a bit more complicated and there are some surprising differences from ODEs. This section will show a few examples of the differences.

Look at the solutions of for different initial history functions.

As long as the initial function satisfies , the solution for is always 1. [Z06] With ODEs, you could always integrate backward in time from a solution to obtain the initial condition.

Investigate at the solutions of for different values of the parameter .

For , the solutions are monotonic, for the solutions oscillate, and for the solutions approach a limit cycle. Of course, for the scalar ODE, solutions are monotonic independent of .

Solve the Ikeda delay differential equation, , for two nearby constant initial functions.
Click for copyable input
Plot the solutions.
Click for copyable input

This simple scalar delay differential equation has chaotic solutions and the motion shown above looks very much like Brownian motion. [S07] As the delay is increased beyond a limit cycle appears, followed eventually by a period-doubling cascade leading to chaos before .

Compare solutions for , , and .
Click for copyable input

Stability is much more complicated for delay equations as well. It is well known that the linear ODE test equation has asymptotically stable solutions if and is unstable if .

The closest corresponding DDE is . Even if you consider just real and the situation is no longer so clear-cut. Shown below are some plots of solutions indicating this.

The solution is stable with and .
Click for copyable input
The solution is unstable with and .
Click for copyable input

So the solution can be stable with and unstable with depending on the value of . A Manipulate is set up below so that you can investigate the plane.

Investigate by varying and .

Propagation and Smoothing of Discontinuities

The way discontinuities are propagated by the delays is an important feature of DDEs and has a profound effect on numerical methods for solving them.

Solve with for .
Click for copyable input
Click for copyable input

In the example above, is continuous, but there is a jump discontinuity in at since approaching from the left the value is , given by the derivative of the initial history function , while approaching from the right the value is given by the DDE, giving .

Near , we have by the continuity of at and so is continuous at .

Differentiating the equation, we can conclude that so has a jump discontinuity at . Using essentially the same argument as above, we can conclude that at the second derivative is continuous.

Similarly, is continuous at or, in other words, at , is times differentiable. This is referred to as smoothing and holds generally for non-neutral delay equations. In some cases the smoothing can be faster than one order per interval.[Z06]

For neutral delay equations the situation is quite different.

Solve with for .
Click for copyable input
Click for copyable input

It is easy to see that the solution is piecewise with continuous. However,

which has a discontinuity at every non-negative integer.

In general, there is no smoothing of discontinuities for neutral DDEs.

The propagation of discontinuities is very important from the standpoint of numerical solvers. If the possible discontinuity points are ignored, then the order of the solver will be reduced. If a discontinuity point is known, a more accurate solution can be found by integrating just up to the discontinuity point and then restarting the method just past the point with the new function values. This way, the integration method is used on smooth parts of the solution, leading to better accuracy and fewer rejected steps. From any given discontinuity points, future discontinuity points can be determined from the delays and detected by treating them as events to be located.

When there are multiple delays, the propagation of discontinuities can become quite complicated.

Solve a neutral delay differential equation with two delays.
Click for copyable input
Plot the solution.
Click for copyable input

It is clear from the plot that there is a discontinuity at each non-negative integer as would be expected from the neutral delay . However, looking at the second and third derivative, it is clear that there are also discontinuities associated with points like , , propagated from the jump discontinuities in .

Plot the second derivative.
Click for copyable input

In fact, there is a whole tree of discontinuities that are propagated forward in time. A way of determining and displaying the discontinuity tree for a solution interval is shown in the subsection below.

Discontinuity Tree

Define a command that gives the graph for the propagated discontinuities for a DDE with the given delays.
Click for copyable input
Get the discontinuity tree for the example above up to .
Click for copyable input
Define a command that shows a plot of and for a discontinuity of order .
Click for copyable input
Plot as a layered graph, showing the discontinuity plot as a tooltip for each discontinuity.
Click for copyable input

Storing History Data

Once the solution has advanced beyond the first discontinuity point, some of the delayed values that need to be computed are outside the domain of the initial history function, and the computed solution needs to be used to get the values, typically by interpolating between steps previously taken. For the DDE solution to be accurate it is essential that the interpolation be as accurate as the method. This is achieved by using dense output for the ODE integration method (the output you get if you use the option InterpolationOrder->All in NDSolve). NDSolve has a general algorithm for obtaining dense output from most methods, so you can use just about any method as the integrator. Some methods, including the default for DDEs, have their own way of getting dense output, which is usually more efficient than the general method. Methods that are low enough order, such as with can just use a cubic Hermite polynomial as the dense output, so there is essentially no extra cost in keeping the history.

Since the history data is accessed frequently, it needs to have a quick lookup mechanism to determine which step to interpolate within. In NDSolve, this is done with a binary search mechanism and the search time is negligible compared with the cost of actual function evaluation.

The data for each successful step is saved before attempting the next step, and is saved in a data structure that can repeatedly be expanded efficiently. When NDSolve produces the solution, it simply takes this data and restructures it into an InterpolatingFunction object, so DDE solutions are always returned with dense output.

The Method of Steps

For constant delays, it is possible to get the entire set of discontinuities as fixed time. The idea of the method of steps is to simply integrate the smooth function over these intervals and restart on the next interval, being sure to reevaluate the function from the right. As long as the intervals do not get too small, the method works quite well in practice.

The method currently implemented for NDSolve is based on the method of steps.

Symbolic Method of Steps

This section defines a symbolic method of steps that illustrates how the method works. Note that to keep the code simpler and more to the point, it does not do any real argument checking. Also, the data structure and lookup for the history is not done in an efficient way, but for symbolic solutions this is a minor issue.

Use DSolve to integrate over an interval where the solution is smooth.
Click for copyable input
Define a method of steps function that returns Piecewise functions.
Click for copyable input
Find the solution for the DDE with .
Click for copyable input
Plot the solution.
Click for copyable input
Check the quality of the solution found by NDSolve by comparing to the exact solution.
Click for copyable input

The method will also work for neutral DDEs.

Find the solution for the neutral DDE with
Click for copyable input
Plot the solution.
Click for copyable input
Check the quality of the solution found by NDSolve by comparing to the exact solution.
Click for copyable input

The symbolic method will also work with symbolic parameter values as long as DSolve is still able to find the solution.

Find the solution to a simple linear DDE with symbolic coefficients.
Click for copyable input

The reason the code was designed to take lists was so that it would work with systems.

Solve a system of DDEs.
Click for copyable input
Plot the solution.
Click for copyable input
Check the quality of the solution found by NDSolve by comparing to the exact solution.
Click for copyable input

Since the method computes the discontinuity tree, it will also work for multiple constant delays. However, with multiple delays, the solution may become quite complicated quickly and DSolve can bog down with huge expressions.

Solve a nonlinear neutral DDE with two delays.
Click for copyable input
Plot the solution.
Click for copyable input
Check the quality of the solution found by NDSolve by comparing to the exact solution.
Click for copyable input


LotkaVolterra Equations with Delay

The LotkaVolterra system models the growth and interaction of animal species assuming that the effect of one species on another is continuous and immediate. A delayed effect of one species on another can be modeled by introducing time lags in the interaction terms.

Consider the system

With no delays, and the system (1) has an invariant that is constant for all and there is a (neutrally) stable periodic solution.

Compare the solution with and without delays.
Click for copyable input

In this example, the effect of even a small delay is to destabilize the periodic orbit. With different parameters in the delayed LotkaVolterra system it has been shown that there are globally attractive equilibria.[TZ08]

Enzyme Kinetics

Consider the system

modeling enzyme kinetics where is a substrate supply maintained at a constant level and molecules of the end product inhibits the reaction step . [HNW93]

The system has an equilibrium when , , and .

Investigate solutions of (1) starting a small perturbation away from the equilibrium.

MackeyGlass Equation

The MackeyGlass equation was proposed to model the production of white blood cells. There are both periodic and chaotic solutions.

Here is a periodic solution of the MackeyGlass equation. The plot is only shown after to let transients die out.
Click for copyable input
Here is a chaotic solution of the MackeyGlass equation.
Click for copyable input
This shows an embedding of the solution in 3D.
Click for copyable input

It is interesting to check the accuracy of the chaotic solution.

Compute the chaotic solution with another method and plot for the difference between computed by the different methods.
Click for copyable input

By the end of the interval, the differences between methods is order 1. Large deviation is typical in chaotic systems and in practice it is not possible or even necessary to get a very accurate solution for a large interval. However, if you do want a high quality solution, NDSolve allows you to use higher precision. For DDEs with higher precision, the method is recommended.

Compute the chaotic solution with higher precision and tolerances.
Click for copyable input
Plot the three solutions near the final time.
Click for copyable input