Mathematica 9 is now available
5.6 Data Loading and Computation

Documentation5. Applications

 

5.6 Data Loading and Computation

Mathematica contains a variety of data loading functions, available through the function Import. This supports many formats such as comma and tab-delimited text data, as well as more specialized formats for graphics, science, sound, and XML. In addition binary data can be loaded using the function Experimental`BinaryImport. If you find that your particular data is not well supported directly by Mathematica, it is possible that you may use a Java API to load the data, which was described in the previous section.

In order to develop data loading technology, it is probably a good idea to work in interactive Mathematica so that you understand how the data loading functions work. When you have done this, you can add data loading to your web applications. At this point you need to determine the source for your data. The following sections discuss some of the possibilities.

5.6.1 File I/O

If your data files are available to the file system of the computer on which your webMathematica server runs, they can be read with a command like Import. For this to work the name and location of the data file must be specified. This can be done in several ways. One way involves placing the files into a directory and setting the full pathname of this directory. This suffers from the extreme disadvantage that if you change the name of the directory you will have to modify all your scripts that use this name. An improvement could be obtained by setting the name of the directory with an initialization parameter. This can be done in MSP.conf with the parameter KernelInitialization as shown in the following.

KernelInitialization=MyApplication`DataDirectory="C:\\Work\\Data"

This assigns the Mathematica symbol MyApplication`DataDirectory to "C:\Work\Data". Note the use of a full context name, this is necessary to prevent the symbol being cleared by the Kernel cleaning mechanism. This can then be loaded in a webMathematica computation as shown in the following example.

<msp:evaluate>
   data = Import[ ToFileName[ MyApplication`DataDirectory], "file.dat", "Table"];
</msp:evaluate>

An alternative is to place the data file into a directory that is on the Mathematica path setting $Path. This is the approach taken by the example Data1.jsp, which is shown below. Another alternative is to place the data file into the same directory as the script, and to use MSPScriptDirectory[], as shown in the following.

<msp:evaluate>
   data = Import[ ToFileName[MSPScriptDirectory[]], "file.dat", "Table"];
</msp:evaluate>

This was used in the XML example Phone.jsp, which was discussed previously. It is particularly convenient to use MSPScriptDirectory[] since it means that data and scripts live in the same directory. Thus the entire web application can be moved from one server to another with a minimum of setting up. One disadvantage is that for JSPs the data file can be loaded by a direct request to the server, thus it should only be used if there is no specialized information present in the data file. This might be the case if only certain information was suitable to be used in a response to each request.

5.6.2 HTTP Upload

Another way to load data into a webMathematica server is to send it from the client machine with the HTTP request. webMathematica contains tools to support this with the function MSPGetUploadFile. This is demonstrated in the example, Upload.jsp, which is shown below.

5.6.3 Database Connection

An important source for data for webMathematica computations is a database. You can interact with a database using the Java database connection toolbox, JDBC, with calls to Java as described above. On Windows you can use the Mathematica Database Access Kit.

5.6.4 Web Service

A last source for data for Mathematica computation is another web site. At present this can be done by opening a URL connection, sending a request and reading the response. This is something that can be done with calls to Java as described above. If the data is available as XML, then it is possible to use XML`Parser`XMLGet, as was described in the section on XML.

5.6.5 Data Examples

The following are a collection of examples of webMathematica working with data.

Loading Data: Load.jsp

If you installed webMathematica as described above, you should be able to connect to this JSP via http://localhost:8080/webMathematica/Examples/Data/Load.jsp. (You may have some other URL for accessing your server.)

This example shows how to load Mathematica packages and how to read a data file that is stored on the server. The data is processed with a data smoothing algorithm and then a couple of plots are made. The source for this page is in webMathematica/Examples/Data/Load.jsp. A section that shows the form tag is shown below.

<form action="Load.jsp" method="post">

<msp:allocateKernel>

<msp:evaluate>
   Needs[ "Graphics`MultipleListPlot`"];
   Needs[ "Statistics`DataSmoothing`"];
</msp:evaluate>

<msp:evaluate>
   data = Flatten[ N[ Import[ "Data/DataFile1.dat"]]];
   term = 4;
   If[ MSPValueQ[ $$term],
      term = MSPToExpression[ $$term]];
</msp:evaluate>



<msp:evaluate>
   dataSmooth = MovingAverage[ data, term];
   MSPShow[ MultipleListPlot[ data, dataSmooth,
         PlotJoined ->{False, True},
         SymbolShape -> Point,
         SymbolStyle -> {{PointSize[0.008], Hue[0]}, {PointSize[0.001]}},
      ImageSize -> 600]]
</msp:evaluate>

<br />
<br />

Number of smoothing terms:
<input type="text" name="term" size="3"
   value="<msp:evaluate>MSPValue[ $$term, "3"]</msp:evaluate>" />

</msp:allocateKernel>

<br />
<br />
<input type="submit" name="btnSubmit" value="Evaluate">
</form>

In this example, an msp:evaluate tag is used to load two Mathematica packages. It is good practice to always load packages in their own tag. The next msp:evaluate tag loads the data set Data/DataFile1.dat using the Import command. This will search for the data file on the Mathematica $Path and will find the Data directory inside of the MSPScripts directory, which by default is located in webMathematica/WEB-INF. It is worth looking inside of your webMathematica web application and confirming that you can locate the data file. The example then computes a moving average and plots the original and the smoothed data. This is all placed inside of a form element so that it is possible to modify the number of terms.

One weakness of this example is that the data has to be loaded from the data file for every computation. It would be better to save the data somehow. In addition it might be useful to allow the data to be uploaded from the client. These will be explored in the sections that follow.

Uploading Data: Upload.jsp

If you installed webMathematica as described above, you should be able to connect to this example via http://localhost:8080/webMathematica/Examples/Data/Upload.html. (You may have some other URL for accessing your server.)

This example allows the user to enter a function to be integrated. The result is then formatted by the typesetting system and saved as an image. The source for this page is webMathematica/Examples/Data/Upload.html and webMathematica/Examples/Data/Upload.jsp. The contents of Upload.html are shown below.

<html>
<head>
<title>Upload File</title>
</head>
<body>
<form method=post enctype="multipart/form-data" action="Upload.jsp">
<p>
Enter a file to upload:
</p>
<input type="file" size=40 name="file">
<br />
<br />
<input type="reset" value="Clear">
<input type="submit" value="Submit">
</form>

<i>
To generate suitable data,
<a href="DataGenerate.jsp">click here</a>
</i>
</body>
</html>

It should first be noted that this is an HTML page; it contains no Java or Mathematica inserts. A servlet container will deliver HTML pages just as it delivers JSPs, and they can all be put together in the same directories inside of the webapp, which can also contain other files such as those containing images or movies. This file contains a single form element that is set up to submit a data file using a enctype attribute of multipart/form-data and an input element of type file. When the submit button is clicked, the form will be submitted along with the file to the Upload.jsp. A selection of Upload.jsp is shown below.

<msp:allocateKernel>

<msp:evaluate>
   Needs[ "Graphics`MultipleListPlot`"];
   Needs[ "Statistics`DataSmoothing`"];
</msp:evaluate>

<msp:evaluate>
   file = "FileName" /. MSPGetUploadFile[];
   data = Flatten[ N[ Import[ file, "Table"]]];
   term = 4;
</msp:evaluate>

<msp:evaluate>
   If[ StringQ[ file],
      dataSmooth = MovingAverage[ data, term];
      MSPShow[ MultipleListPlot[ data, dataSmooth,
         PlotJoined ->{False, True},
         SymbolShape -> Point,
         SymbolStyle -> {{PointSize[0.008], Hue[0]}, {PointSize[0.001]}},
         ImageSize -> 600]]]
</msp:evaluate>

</msp:allocateKernel>

In this selection we see how the MSP function MSPGetUploadFile is used to retrieve the name of the datafile by which the data has been stored on the server. This filename can then be used to read the data as was done in the previous example and the computation can proceed. It should be noted that this solution is somewhat limited; the data is only available once for one computation, and no facility has been added to change a parameter, for example, to control the data smoothing. For this the data needs to be stored in a session variable. This is explored in the next example.

In this example a test was made to determine whether the filename was in fact a string. This checks that a data file has in fact been uploaded. A file would not be uploaded if the page Upload.jsp was visited directly instead of coming from a request to Upload.html.

MSPGetUploadFile returns a list of useful information, including the filename that is used on the server, the original filename used on the client, and the content-type. In a case where there are multiple files to be uploaded, MSPGetUploadFile will throw an exception. If you wish to upload more than one file, you can use MSPGetUploadFileList.

Session Storage of Data: Session.jsp

If you installed webMathematica as described above, you should be able to connect to this JSP via http://localhost:8080/webMathematica/Examples/Data/Session.html. (You may have some other URL for accessing your server.)

This example allows the user to enter a function to be integrated. The result is then formatted by the typesetting system and saved as an image. The source for this page is webMathematica/Examples/Data/Session.html, webMathematica/Examples/Data/Session.jsp, and webMathematica/Examples/Data/SessionProcess.jsp. The contents of Session.html are shown below.

<html>
<head>
<title>Upload File</title>
</head>
<body>
<form method=post enctype="multipart/form-data" action="Session.jsp">
<p>
Enter a file to upload:
</p>
<input type="file" size=40 name="file">
<br />
<br />
<input type="reset" value="Clear">
<input type="submit" value="Submit">
</form>

<i>
To generate suitable data,
<a href="DataGenerate.jsp">click here</a>
</i>
</body>
</html>

This is very similar to Upload.html, except that it calls the JSP Session.jsp when the form is submitted. Session.jsp is shown below.

<%@ page language="java" %>
<%@ taglib uri="/webMathematica-taglib" prefix="msp" %>

<msp:allocateKernel>

<msp:evaluate>
   file = "FileName" /. MSPGetUploadFile[];
   data = Flatten[ N[ Import[ file, "Table"]]];
   MSPSessionVariable[ UploadedData, Null];
   UploadedData = data;
</msp:evaluate>

<jsp:forward page="SessionProcess.jsp" />

</msp:allocateKernel>

This short JSP reads the data that was uploaded, and then uses MSPSessionVariable to store the data in a session variable named UploadedData. This is a very convenient way to store data persistently from one HTTP request to another. The data is actually stored in the server, so that even if the Mathematica kernel was restarted the data would still be available. The storage uses what is known as an HTTP session and the server may use cookies or some other mechanism to actually store the data. This is how a shopping cart in an e-commerce web site works. As far as a developer of a webMathematica web site is concerned, it is all very simple; just use the function MSPSessionVariable, giving it the name of the variable and an initial value. After the data has been read, there is a jsp:forward to the page that actually does the computations and plots, Data3.jsp. Often it is a very good design principle to divide the code over a number of separate pages that all do different tasks; if you develop pages that have large amounts of complicated code in them, then refactoring might improve your system. A selection of SessionProcess.jsp is shown below.

<form action="SessionProcess.jsp" method="post">

<msp:allocateKernel>

<msp:evaluate>
   Needs[ "Graphics`MultipleListPlot`"];
   Needs[ "Statistics`DataSmoothing`"];
</msp:evaluate>

<msp:evaluate>
   MSPSessionVariable[ UploadedData, Null];
   term = 4;
   If[ MSPValueQ[ $$term],
      term = MSPToExpression[ $$term]];
</msp:evaluate>

<msp:evaluate>
   If[ UploadedData =!= Null,
      dataSmooth = MovingAverage[ UploadedData, term];
      MSPShow[ MultipleListPlot[ UploadedData, dataSmooth,
         PlotJoined ->{False, True},
         SymbolShape -> Point,
         SymbolStyle -> {{PointSize[0.008], Hue[0]}, {PointSize[0.001]}},
      ImageSize -> 600]]]
</msp:evaluate>
<br /><br />
Number of smoothing terms:
<input type="text" name="term" size="3"
   value="<msp:evaluate>MSPValue[ $$term, "3"]</msp:evaluate>" />

</msp:allocateKernel>
<br /><br />
<input type="submit" name="btnSubmit" value="Evaluate">
</form>

SessionProcess.jsp is very similar to Load.jsp; the main difference is the line that obtains the data. Instead of reading data from a file, it uses the session variable UploadedData, which was assigned to the data in the previous JSP, Session.jsp. If, by some chance, UploadedData was not previously defined, as might happen if SessionProcess.jsp is visited directly, then UploadedData will have the value Null. Here this just prevents any plot from being produced, but in general it might be used to transfer to an error page.

A further change to this section might be to store the data permanently on the server. For a simple data file, this could easily be done by using Mathematica file output operations. A more sophisticated application could use Java database connectivity to store the data and certain related information.

Session variables are discussed further in Advanced Topics: Variables.



Any questions about topics on this page? Click here to get an individual response.Buy NowMore Information
THIS IS DOCUMENTATION FOR AN OBSOLETE PRODUCT. CURRENT WEBMATHEMATICA DOCUMENTATION IS NOW AVAILABLE.