2.14.4 Memory Management
Finding memory usage.
Particularly for symbolic computations, memory is usually the primary resource which limits the size of computations you can do. If a computation runs slowly, you can always potentially let it run longer. But if the computation generates intermediate expressions which simply cannot fit in the memory of your computer system, then you cannot proceed with the computation.
Mathematica is careful about the way it uses memory. Every time an intermediate expression you have generated is no longer needed, Mathematica immediately reclaims the memory allocated to it. This means that at any point in a session, Mathematica stores only those expressions that are actually needed; it does not keep unnecessary objects which have to be "garbage collected" later.
This gives the number of bytes of memory currently being used by Mathematica.
In:= MemoryInUse[ ]
This generates a 10000-element list.
In:= Range // Short
Additional memory is needed to store the list.
In:= MemoryInUse[ ]
This list is kept because it is the value of Out. If you clear Out, the list is no longer needed.
In:= Unprotect[Out]; Out=.
The memory in use goes down again.
In:= MemoryInUse[ ]
This shows the maximum memory needed at any point in the session.
In:= MaxMemoryUsed[ ]
One issue that often comes up is exactly how much memory Mathematica can actually use on a particular computer system. Usually there is a certain amount of memory available for all processes running on the computer at a particular time. Sometimes this amount of memory is equal to the physical number of bytes of RAM in the computer. Often, it includes a certain amount of "virtual memory", obtained by swapping data on and off a mass storage device.
When Mathematica runs, it needs space both for data and for code. The complete code of Mathematica is typically several megabytes in size. For any particular calculation, only a small fraction of this code is usually used. However, in trying to work out the total amount of space available for Mathematica data, you should not forget what is needed for Mathematica code. In addition, you must include the space that is taken up by other processes running in the computer. If there are fewer jobs running, you will usually find that your job can use more memory.
It is also worth realizing that the time needed to do a calculation can depend very greatly on how much physical memory you have. Although virtual memory allows you in principle to use large amounts of memory space, it is usually hundreds or even thousands of times slower to access than physical memory. As a result, if your calculation becomes so large that it needs to make use of virtual memory, it may run much more slowly.
MemoryConstrained works much like TimeConstrained. If more than the specified amount of memory is requested, MemoryConstrained attempts to abort your computation. As with TimeConstrained, there may be some overshoot in the actual amount of memory used before the computation is aborted.
Finding the size of expressions.
Although you may find ByteCount useful in estimating how large an expression of a particular kind you can handle, you should realize that the specific results given by ByteCount can differ substantially from one version of Mathematica to another.
Another important point is that ByteCount always gives you the maximum amount of memory needed to store a particular expression. Often Mathematica will actually use a much smaller amount of memory to store the expression. The main issue is how many of the subexpressions in the expression can be shared.
In an expression like f[1 + x, 1 + x], the two subexpressions 1 + x are identical, but they may or may not actually be stored in the same piece of computer memory. ByteCount gives you the number of bytes needed to store expressions with the assumption that no subexpressions are shared. You should realize that the sharing of subexpressions is often destroyed as soon as you use an operation like the /. operator.
Nevertheless, you can explicitly tell Mathematica to share subexpressions using the function Share. In this way, you can significantly reduce the actual amount of memory needed to store a particular expression.
Optimizing memory usage.
On most computer systems, the memory used by a running program is divided into two parts: memory explicitly allocated by the program, and "stack space". Every time an internal routine is called in the program, a certain amount of stack space is used to store parameters associated with the call. On many computer systems, the maximum amount of stack space that can be used by a program must be specified in advance. If the specified stack space limit is exceeded, the program usually just exits.
In Mathematica, one of the primary uses of stack space is in handling the calling of one Mathematica function by another. All such calls are explicitly recorded in the Mathematica Stack discussed in Section 2.6.12. You can control the size of this stack by setting the global parameter $RecursionLimit. You should be sure that this parameter is set small enough that you do not run out of stack space on your particular computer system.