Statistics`DataSmoothing`
A series of raw data is sometimes transformed to a new series of data before it is analyzed. The purpose of this transformation is to smooth out local fluctuations in the raw data, so the transformation is called data smoothing or a smoother. One common type of smoother employs a linear transformation and is called a linear filter. A linear filter with weights transforms the given data to weighted averages for . Notice that the new data set has length . If the linear filter is also called an term moving average. If all the weights are equal and they sum to unity, the linear filter is called a simple moving average.
If the median of data points is substituted in place of the average, we obtain a moving median smoother of span . Repeated application of a moving median smoother can quickly lead to convergence for odd . Since the values at the ends of the data series are lost with each application of the smoother, we maintain the length of the series by adding on the end values from the original data at all the intermediate steps of repeated moving median smoothing.
Exponential smoothing corresponds to a weighted average with exponentially decreasing weights. Let be the smoothed value at time , then can be obtained efficiently by using where is the smoothing constant (). An initial value is needed to start the iteration and it is commonly chosen to be . For multivariate data the smoothing constant can be a matrix.
Data smoothing functions.
The functions in this package work for both univariate and multivariate data. Data should be entered as a list in the univariate case, and a list of length lists in the variate case.
This loads the package.
In[1]:= <<Statistics`DataSmoothing`
Data must be entered as a list.
In[2]:= data1 = {7.,4.,5.,3.,8.,5.,2.,3.,6.,8.};
Here is a simple 2term moving average.
In[3]:= MovingAverage[data1, 2]
Out[3]=
In the case of multivariate data, each entry in a data list is itself a list.
In[4]:= data2 = {{a,b}, {c,d}, {e,f}, {g,h}};
This gives a simple 3term moving average of the twodimensional data. The moving average is done for each dimension (or column) of the data separately.
In[5]:= MovingAverage[data2, 3]
Out[5]=
A moving median is an example of nonlinear smoothing. It is often used to preserve edges while eliminating noisy spikes in the data. This smooths the data using a moving median of span 3.
In[6]:= MovingMedian[data1, 3]
Out[6]=
Here smoothing is repeated until the result no longer changes. Only oddnumbered spans are allowed in repeated smoothing to ensure convergence.
In[7]:= MovingMedian[data1, 3, RepeatedSmoothing>True]
Out[7]=
Here is an example of smoothing multivariate data by a moving median. The moving median is done for each dimension (or column) of the data separately.
In[8]:= MovingMedian[{{3,4},{9,2},{1,5},{2,6},{2,3}}, 4]
Out[8]=
A simple moving average is a special case of a linear filter where all the weights are equal and their sum is unity.
In[9]:= LinearFilter[data1, {1/2, 1/2}]
Out[9]=
This linear filter has weights and gives . Notice the order in which the weights are entered.
In[10]:= LinearFilter[data1, {1, 2, 3}]
Out[10]=
For multivariate data you can include matrix weights for the linear filter. Here the weights are , and .
In[11]:= LinearFilter[data2, {{{1,0.5},{0.5,1}},{{1,0},{0,1}}}]
Out[11]=
Here exponential smoothing is done with the smoothing constant 0.8 and starting value 0.5.
In[12]:= ExponentialSmoothing[data1, 0.8, 0.5]
Out[12]=
This time the starting value is taken to be the first entry in the data list, which is 7.
In[13]:= ExponentialSmoothing[data1, 0.8]
Out[13]=
