Advanced Manipulate Functionality

This tutorial covers advanced features of the Manipulate command. It assumes that you have read "Introduction to Manipulate" and thus have a good idea what the command is for and how it works overall.
This tutorial also, in places, assumes a familiarity with the lower-level dynamic mechanism covered in "Introduction to Dynamic" and "Advanced Dynamic Functionality".
Please note that this is a hands-on tutorial. You are expected to actually evaluate each input line as you reach it in your reading, and watch what happens. The accompanying text will not make sense without evaluating as you read.
Controlling Automatic Reevaluation
Some Manipulate examples "spin", continually reevaluating their contents even when no sliders are being moved. Sometimes this is in fact exactly what you intend. For example, here is a droopy triangle, which always sags down in the middle. You can drag it back up using the slider, but as soon as you stop moving, it starts falling down again.
This happens because the variable y is being changed by the code in the first argument, and the system, correctly and helpfully, notices that since the value of y has changed, the contents need to be evaluated and displayed again, which in turn causes the value of y to change again, and so on, until, in this case, we reach a stable point at y=-1. After that the value of y no longer changes, and the contents are no longer continually redrawn, until you touch the slider again. (If you have a CPU activity monitor on your system you can verify that while the triangle is drooping, the Wolfram System is using CPU time, but once it reaches the bottom, the Wolfram System's CPU usage stops.)
Of course it is possible to construct examples that do not stop. Here we declare two variables, both initialized to zero, and include code in the body of the Manipulate to continuously update the value of one of them based on the value of the other.
Any time the step size is moved away from zero, the content area will continually update, and a CPU monitor will indicate that the Wolfram System is using CPU time. This will go on for as long as you let it. (Fortunately it does not totally consume the CPU, and other activities in the front end are not hindered by this activity; you can keep editing, evaluating, etc., while it is running. You may think of it sort of like an animated image or applet running in a web browser.)
In some cases, however, the continual reevaluation is pointless and undesirable. Consider this somewhat contrived example: any time it is present on screen (i.e. in an open window and not scrolled offscreen to the point where no part of it is visible), it will constantly reevaluate itself, consuming CPU time even though nothing is changing.
This happens because the variable temp has its value changed during the evaluation, even if the value of n has not changed (i.e. it is reset to two different values each time through). The spinning is pointless because the value of temp is set before it is ever used.
Another way to get inadvertent and pointless spinning is to make a function definition or other complex assignment in the body of the Manipulate, as in this example.
In both these cases, the problem can be solved by making the offending variables be local variables inside a Module. (This is good programming practice in any case, quite aside from any desire to avoid pointless updating.)
Nothing you do to local Module variables will cause retriggering, because it is part of the definition of Module that values do not survive from one invocation to the next (therefore the result will not be any different the next time around just because of anything done to the value of a local variable during the current cycle).
Another solution is to use the TrackedSymbols option of Manipulate to control which variables are allowed to cause updating behavior. The default value, Full, means that any symbols that appear explicitly (lexically) in the first argument will be tracked. (This means, among other things, that temporary variables and other such problems inside the definitions of functions you use in your Manipulate example will not cause infinite reevaluation problems, because they do not occur explicitly in the first argument, only indirectly through functions you call.)
Taking the second example, if for some reason you do not want f to be a local Module variable, and you cannot move its definition outside the Manipulate (there are sometimes good reasons for both those conditions, in more complicated cases), you can use TrackedSymbols to disable updating triggered by f:
This example updates the content area only when n changes its value as a result of moving the slider.
The subject of when exactly a given dynamic expression will be updated is complex, and is addressed in "Introduction to Dynamic" and "Advanced Dynamic Functionality". In reading those, keep in mind that Manipulate simply wraps its first argument in Dynamic and passes the value of its TrackedSymbols option to a Refresh inside that. Everything to do with updating is handled by that Dynamic and Refresh.
Nesting Manipulate
You can put one Manipulate inside another. For example, here we use a slider in an outer Manipulate to control the number of sliders in the inner Manipulate.
While nesting Manipulate many levels deep is possible, and will work, it is probably not the most useful feature in the world. But by nesting once, you have in effect created a parameterized user interface construction interface. The outer Manipulate allows you to control parameters that determine the user interface presented by the inner Manipulate. With some slightly more complex programming than in the previous example, remarkable things can be done.
Interdependent Controls
It is possible to make the range of one slider in a Manipulate depend on the position of another slider. For example, the function Binomial[n,m] makes sense only when m<=n, so you might want to make an m-slider whose range is from 1 to the current value of n. You can do this simply by using n in the variable specification for m, like this.
Note that if you first move both sliders part way towards the right, then move the n-slider left, the m-slider will automatically move to the right, because its maximum is getting smaller. If you move n far enough to the left, to the point where it becomes smaller than the current value of m, the m-slider will display a red "out of range" indicator, because m is now larger than its maximum allowed value.
You might wonder why m is not automatically reset to the current maximum, when the maximum is set lower than its current value. The reason is that sometimes it is preferable to leave the value alone, and if you want to have it reset automatically, it is easy to do manually. For example, you can add an If statement to the code in the first argument.
Generally speaking, you can use Manipulate variables in the definition of other variables without restriction, though it is certainly possible in this way to create peculiar interactions that are more confusing than helpful.
This example shows another variation, using a check box to control the range of a slider: something like this can be useful in cases where you want to provide fine and coarse ranges, for example.
Dealing with Slow Evaluations
Manipulate does not precompute all the possible output values you could reach by moving its sliders: that would be completely impractical for all but the most trivial cases. That means it has to calculate, format, and display the current value in real time as each slider is being dragged. Obviously no matter how fast your computer, there is a limit to how much computation can be done in a finite amount of time, and if the expression you use in the first argument to Manipulate takes more than about a second to evaluate, you will not have a very satisfactory experience using the Manipulate.
Many very interesting and powerful computations can be done in under a second, and as computers get faster the range will only increase (people are not getting any faster, so the amount of time available before the example seems too sluggish should remain unchanged for quite a while). But some computations just cannot be done that fast, and some alternative is necessary if you want to use them within Manipulate. Fortunately there are several good ways of dealing with slow evaluations.
For purposes of this section we are going to use Pause to simulate a slow evaluation. The main reason for this is that any actual computation would run at such widely differing speeds on different users' computers that it would be hard to illustrate the point with any one example. So where you see a Pause command, please imagine that something terribly complex and interesting is being done, resulting in a fantastically detailed and enlightening output.
To get a feel for the problem, try dragging the following slider. While this example is not unacceptable, it is on the borderline of something not worth playing with. If the delay is increased to several seconds, it becomes quite pointless. (And beyond five seconds you will start seeing $Aborted instead of the number, because the system is protecting itself from unreasonably long evaluations, which block other activity in the front end in this situation.)
The simplest improvement is to add the option ContinuousAction->False to the Manipulate.
In this example the slider moves smoothly and instantaneously as it is dragged, but the value in the output area does not attempt to track in real time. Instead it updates only when the slider is released.
A more subtle difference is that when the value updates in this example, it does so without blocking other activity in the front end. You can see this by the fact that the cell bracket becomes outlined for a second each time the slider is released, and you can continue typing or doing other work in the front end during that second. There is no five-second limit to such nonblocking evaluations, so by using the ContinuousAction->False option, arbitrarily long evaluations can be used. (Though something that takes a minute is still probably better done as a normal Shift+Return evaluation than inside Manipulate.)
A more sophisticated alternative is to use the ControlActive function to present an alternative, simpler and faster display while the slider is being dragged, and do the long computation only when it is released.
ControlActive takes two arguments: the first is returned if the expression is evaluated while a control (e.g. a slider) is currently being dragged with the mouse, and the second if no control is currently active. (See the documentation for ControlActive for some fine print about exactly when which argument is returned.)
In this example we use just x as the preview to be displayed while the slider is being dragged, and x with a box around it, plus a second of delay, as the final display to be presented when the slider is released. Note that we have removed the ContinuousAction->False option from the example above.
Note that the cell bracket is outlined, indicating a nonblocking evaluation, only when the slider is released. While the slider is being dragged, evaluations are done in a blocking way for maximum interactive performance.
Here is a slightly more realistic example of where ControlActive can be useful. This example shows how the default behavior of DensityPlot is to use fewer sample points while the slider is being dragged.
But even the higher number used after the slider is released is not enough to produce a satisfactory plot. If we just set a fixed, larger number of plot points, the result is pretty, but interactive performance is not good enough.
(There is still a difference between the active and inactive forms, because by default several different options, not just PlotPoints, depend on ControlActive.)
The optimal combination of speed and quality can be achieved by using ControlActive explicitly in the value of the PlotPoints option.
The result is an example that displays a crude, but virtually instantaneous, preview of the graphic, then spends many seconds constructing a high-resolution version when the slider is released.
The next section explains a more complex solution that works in cases where some changes to the parameters require a slow evaluation while others could update the display much more rapidly.
Using Dynamic inside Manipulate
You might want to read "Introduction to Dynamic" before finishing this section, as it refers to the use of explicit Dynamic expressions, which are not explained in this tutorial.
When you move the sliders (or other controls) in a Manipulate, the expression given in the first argument is reevaluated from scratch for each new parameter value. "Dealing with Slow Evaluations" discusses a number of general things that can be done if evaluation of this first argument is too slow to allow smooth interactive performance of the Manipulate. But in some cases it is possible to separate the evaluation into slower and faster parts, and thereby achieve much better performance.
Consider this example where one slider controls the contents of a 3D plot, while the other controls its viewpoint.
When the n-slider is moved, it is obviously necessary to recompute the 3D plot, because it actually changes shape. The plot becomes jagged while the slider is being dragged, then improves shortly after you release it, which is correct and as expected. When you move the v-slider, on the other hand, there is no point in recomputing the function, because only the viewpoint has changed. But Manipulate has no way of knowing this (and in more complex cases it is genuinely impossible for this kind of distinction to be made in any automatic way), so the whole plot is regenerated from scratch each time v is changed.
To improve this example, we can tell Manipulate that the ViewPoint option should be updated separately from the rest of the output, which we can do by wrapping Dynamic around the right-hand side of the option.
Notice that now when the v-slider is moved, the plot does not revert to the jagged appearance, and actually rotates faster than before. This is because the plot is no longer being regenerated with each movement.
To explain exactly why this works requires an understanding of the internals of the Dynamic mechanism explained in "Introduction to Dynamic" and "Advanced Dynamic Functionality". In short, Manipulate always wraps Dynamic around the expression given in its first argument, and normally any changes to variables used in the first argument will trigger updates of that Dynamic. But when a variable occurs only inside an explicit Dynamic nested inside the one implicitly created by Manipulate, an update of the outer Dynamic will not be triggered, only an update of the inner Dynamic in which it resides.
Explaining the full range of what is possible by using Dynamic explicitly inside Manipulate is beyond the scope of this document, but another common case worth looking at involves a situation where the slow part of some computation involves only some of the input variables.
In the next example we construct a large table of numbers (using RandomReal in this case, but in a real-world example it might be a much more complicated, slower computation, or even one involving reading external data from the network). After constructing the data, we display it using a fairly simple, fast function (illustrated here by just raising the coordinate values to a power).
Note that when the n-slider is moved, the number of points changes, and they jump around because a new random set is generated each time. But when the p-slider is moved, updates are smoother, and the points do not jump around. This is because the inner Dynamic wrapped around the use of p prevents the first argument as a whole from being reevaluated. Thus no new random points are generated only the presentation of the existing ones is updated.
The option SynchronousUpdating->False is used to cause the outer Dynamic (the one implicitly created by Manipulate) to update asynchronously (visible by the fact that the cell bracket becomes outlined when the n-slider is moved). Asynchronous updating does not give quite as smooth updating, but if the evaluation takes a long time, it does not block other activity in the front end.
The inner Dynamic uses the default synchronous updating, so when the p-slider is moved, updating is smooth and rapid.
It is thus practical, using the technique illustrated here, to make an example that takes many seconds, even minutes, to respond when one slider is changed, yet preserve rapid interactive performance when other controls, which do not require the long computation to be repeated, are changed.
You can also use Dynamic inside Manipulate to make the output dynamically respond to things other than the values of the Manipulate control variables. For example, here is an example taken from an earlier section, except that we have made it respond dynamically to the current mouse position.
Any time the mouse is over the area of the plot, the center of the lines will follow it (without a click). Consult the documentation for MousePosition for further detail.
It is important to remember also that Manipulate is not the only way of creating interactive user interfaces in the Wolfram Language. Manipulate is intended to be a simple, yet powerful, tool for defining user interfaces at a very high level. But when you reach the limits of what it is capable of doing, either in terms of control layout, updating behavior, or interaction with external systems, it is always possible (and often not terribly difficult) to drop to a lower level of interface programming using functions such as Dynamic and EventHandler.
Dynamic Objects in the Control Area
We saw in "Introduction to Manipulate" that it is possible to add a variety of elements to the control area of a Manipulate, for example titles and delimiters, as in this example.
There is in fact virtually no limit to what can be put into the controls area, including arbitrary formatting constructs and dynamic objects, even controls that are not part of the Manipulate control specifications. Anything placed in the variables sequence that is either a string or has head Dynamic, Style, or ExpressionCell will automatically be interpreted as an annotation to be inserted in the controls area.
You might want to read "Introduction to Dynamic" before finishing this section, as we refer to the use of explicit Dynamic expressions, which are not explained in this tutorial.
Suppose you want to show plots of the individual x and y sine functions that combine to form the Lissajous figure. You could do this by putting all three functions into the output area, using Grid to lay them out.
There is in fact a lot to be said for this presentation. But suppose you instead want to leave the main output area as it is, with a large, prominent presentation of the Lissajous figure itself, and show the individual sine functions only much smaller, in association with the controls for each direction. You can do this by placing a dynamic plot object into the controls area, as follows.
Just as the headings in the example at the start of this section were mixed in with the controls simply by listing them in the variable specifications sequence, here we have placed dynamically updated plots in the variable specification sequence. Dynamic is used explicitly in these subplots so that they will update when the controls are moved. (The main output area does not need an explicit Dynamic because Manipulate automatically wraps Dynamic around its first argument.)
It is worth briefly discussing here why it is that an example like this can just work. The reason is that the output of Manipulate is not a special, fixed object that just connects a set of controls with a single output area. Instead, the output of Manipulate is built up using the same formatting, layout, user interface, and dynamic interactivity features that can be accessed at a lower level using the techniques discussed in "Introduction to Dynamic" (which in fact includes an example of how to build up a simple version of Manipulate by hand). In some ways the relationship between Manipulate and the lower level interactive features is like the relationship between Plot and Graphics. The result of evaluating a high-level Plot command is a low-level Graphics object, and if Plot is not able to generate the specific graphic you want, you are always free to use Graphics directly. You can use the Prolog and Epilog options to add arbitrary graphical elements to a Plot. Furthermore there is no graphical output you can get using Plot that you cannot get using Graphics. Plot has no special access to any features in the Wolfram Language unavailable at the lower level.
Likewise, Manipulate does not have any special access to features unavailable with lower level functions: there is nothing you can do with Manipulate that you cannot do with Dynamic, it is just a higher-level, more convenient function for building a certain style of interface.
So when you use dynamic objects in the control labels, as in the example above, you're just adding a couple more Dynamic objects to the already fairly complex set of Panel objects, Grid objects, Dynamic objects, and DynamicModule objects that constitutes the output of a Manipulate command. There is nothing really different there, just more of it, so it should come as no surprise that the new dynamic elements interoperate smoothly with the others.
Although it is not always sensible to do so, it is possible to build completely arbitrary interactive dynamic user interfaces entirely in the controls area of a Manipulate.
Arbitrary Layout of Controls in the Control Area
Even with the techniques described in the previous section for including arbitrary dynamic content in the control area of Manipulate, the layouts within the control area have, so far, all been very simple. That's because the ControlPlacement option only allows you to pick a side for each control or annotation. Once you've chosen your placements, the controls and annotations are simply stacked into columns on the chosen sides.
The simplicity of the default layout illustrates an important design principle of Manipulate. Manipulate allows you to largely ignore details that interface programmers typically worry about, like the detailed placement of controls and annotations, and instead focus on giving a high-level description of an interface very quickly, immediately creating an interface that works.
However, when you do choose to focus on these details, there are features that allow arbitrarily detailed layout of controls.
First, recall from the previous section that you can use Style as an argument to Manipulate, for example to apply font options to a piece of text. Here's the example again.
Style is not the only function supported in this way. All of the Wolfram Language's layout and styling constructs are supported directly as arguments to Manipulate. Layout functions like Grid, Row, Framed, Item, Panel, and Text are supported, as are TabView, OpenerView, PaneSelector, and other view-like constructs. See "Layout & Tables" for information about all these useful objects.
There is one other construct, Control, which is important to understand when discussing arbitrary layout of controls in Manipulate. Control makes it possible to use all these layout constructs in Manipulate, without giving up the nice compact Manipulate syntax for specifying variables and controls.
In its simplest form, the Control wrapper looks completely redundant.
In fact, in that case, the Control wrapper is optional. There's no difference in the behavior or display of those two interfaces. And notice that the argument to Control is the familiar list containing a local variable and its domain, just like every other Manipulate variable you're used to seeing.
The real power of Control, though, is that it can be used anywhere inside the arguments to Manipulate, including inside those layout and styling constructs mentioned above. So if you want to position a set of controls horizontally rather than vertically, you can do so by constructing a Row that contains a list of Control expressions, each of which continues to use the compact, high-level specification for the controls.
Note that the Control wrappers are not mere syntactic sugar in this case. Without those wrappers, Manipulate would not have known that {u,{True,False}} was a control specification. Instead, when it encountered {u,{True,False}} inside of Row, it would have just evaluated and displayed it like any other element of Row, and the u would have been treated as a global variable, rather than as a local Manipulate variable.
Coming back to our original example, here it is reworked with OpenerView, which is used for its ability to both group and hide sets of controls, along with Column and Control. Also added is a row of controls for adjusting the plot's axes and size.
Note that this reworked code is more than doubled in size compared to the original. This is no longer a lightweight, high-level description of an interface. It specifies details that typical uses of Manipulate would completely ignore. There is definitely a trade-off to doing so.
In practice, you should carefully consider these trade-offs when thinking about the layout needs of your interface. You may find that using Control and other layout constructs in Manipulate is still more convenient than having to build a Dynamic interface from scratch. Or, you may even find it useful to use Control outside Manipulate, to take advantage of this nice compact syntax for creating controls even when building custom interfaces.
Custom Control Appearances
You might want to read "Introduction to Dynamic" before reading this section, as we refer to the use of explicit Dynamic expressions, which are not explained in this tutorial.
Suppose you want to use a type of control that is not supported by Manipulate, for example one you have built yourself using graphics and dynamics. Here is a block of code that defines a custom style of slider, one that shows its value at the thumb position. Do not worry about understanding the details of how this code works, though it is not overly complicated beyond the details of drawing the desired elements in the right places.
Here is an example of what this new control looks like: click anywhere to move the thumb around, just like with a normal slider.
To use a custom control in Manipulate, you include a pure function that is used to generate the control object as part of the variable specification. As long as your function conforms to the convention used by all the built-in control functions, with the variable (inside Dynamic) as the first argument and the range as the second argument, you can simply use the function name and the appropriate arguments will be passed to it automatically by Manipulate. Here we see our custom control used in a simple Manipulate.
(The notation ## means that all the arguments will be passed in to the function, not just the first one.)
Note that if you supply the necessary information in the pure function, you do not have to specify the min and max as part of the variable specification.
However, if you do that, then Manipulate is not aware of the range you chose to use in the slider, which means that the very nice autorun feature (as described in the documentation for Manipulate) cannot work. So generally it is a good idea to include the range in the variable specification, and let the control function inherit it.
Naturally it is possible to combine standard and custom controls freely; here we use two of our new sliders together with a SetterBar supplied automatically by Manipulate.
It is also possible to combine custom controls with other dynamic elements in the controls area (discussed in the previous section).
This example should give some idea of how far Manipulate can be pushed to create complex interfaces. However, it is important to remember that Manipulate is not the only way to create interfaces in the Wolfram Language. "Introduction to Dynamic" provides further information and examples showing how to create free-form interfaces not restricted to the model provided by Manipulate.
One of the nice things about building an interface like this inside Manipulate is that it lets you use Autorun (click the plus icon in the top right corner of the panel and choose Autorun) to put the example through its paces, varying each variable according to a sensible interpolating pattern.
On the other hand, Manipulate restricts you to a certain set of layouts and behaviors which, while very flexible and expandable, are still fixed compared to what is possible using the lower level features described in "Introduction to Dynamic".