Web Services

Web services are typically function calls made over a network using web technology to transmit the information. Often the information that is transmitted is formatted as XML.

A web service can be contrasted with a typical web request even though they both use web technology. A typical request is started by a human who uses a web browser and the result is HTML that is displayed in the browser. In a web service the request is started from a program and the result comes back to the program. In some cases the program is running in a browser, and it will modify the HTML that the browser is displaying. This is a common usage for AJAX applications.

It is often convenient to use the terms client and server when discussing web services. The client makes a request to the server which responds with a result. Note that the client and server need not be on the same machine and might be different types of computers. Also, the client and server might be running different languages.

Web services can be informal ad hoc services where both ends agree on a format for data. Alternatively, there are web services specifications such as SOAP. These define the format for information transmitted for the web service. They may also provide other features such as being able to query a server to find out details of the web service that the server is publishing.

Mathematica can work as a client for web services. The HTTP and XML tools are good for informal web services, while the Web Services Link, allows you to call to use SOAP web services. Here, you can see how to use webMathematica as a server for web services.

Informal Web Services

Informal web services do not use a special protocol, and this means that a client needs to know details of how the server passes data. The advantage of an informal web service is that it is easy to set up. One use of an informal web service is for a program running inside a browser to make a call to the server. Examples are a JavaScript program using AJAX or an Actionscript program running in the Flash runtime. In fact, the webMathematica interactive tools, based on MSPManipulate, use an informal web service to send details of the user interface that is to be constructed to the Flash runtime.

AJAX Example

This example uses AJAX technology to demonstrate the setting up of an informal web service. It will not discuss all the details of how AJAX works, which is covered in the section on using AJAX with webMathematica.

The source for this example can be found in the webMathematica web application in the directory Examples/AJAX (the full path in Tomcat would be webapps/webMathematica/Examples/AJAX). The two files are LoadXML.jsp, which contains the AJAX code to call to the server, and ReturnXML.jsp, which returns the result from the server. If your server is configured and running, you can test this example with the URL http://localhost:8080/webMathematica/Examples/AJAX/LoadXML.jsp. (You may have some other URL for accessing your server.)

First, this is the source of ReturnXML.jsp. This sets the content type to be text/xml, and then takes the input number, $$num, and does an integer factorization of the factorial of the input. The result is then formatted into XML, making use of symbolic XML.

<%@ page contentType="text/xml"%> 
<%@ taglib uri="http://www.wolfram.com/msp" prefix="msp" %>

<msp:evaluate>
num = MSPToExpression[ $$num];
factors = FactorInteger[ num!];
xml =
  XMLElement[ "factors", {},
     Flatten[Apply[
          {XMLElement["factor", {}, {ToString[#1]}],
         XMLElement["exponent", {}, {ToString[#2]}]} &, factors, {1}]]];
ExportString[ xml, "XML"]
</msp:evaluate>

You can test this with the URL http://localhost:8080/webMathematica/Examples/AJAX/ReturnXML.jsp?num=8. You should see something like the following.

<factors> 
 <factor>2</factor> 
 <exponent>7</exponent> 
 <factor>3</factor> 
 <exponent>2</exponent> 
 <factor>5</factor> 
 <exponent>1</exponent> 
 <factor>7</factor> 
 <exponent>1</exponent> 
</factors>

This is a simple XML fragment made up for this example. The root tag is factors and it contains sequences of factor and exponent tags. This might be improved if each pair of factor and exponent tags were placed in a containing tag. The current approach is taken to keep the code simple.

Note how the XML is created by first making an XMLElement expression, and then converting this to a textual representation of the XML with ExportString. An alternative would be to use lots of string concatentation operations. It is very strongly recommended to use the symbolic XML technology of XMLElement. Symbolic XML will be much more robust, since a coding error will not lead to a string of ill-formed XML. It will also be much more flexible, if you want to change one of the tags it will be easier. Also, it is convenient to use other XML features such as attributes or namespaces. Mathematica provides some very nice tools for working with XML and it is extremely good to use them.

Here is part of the JavaScript function in LoadXML.jsp. It shows how the XML result is extracted from the HTTP result and arrays of factor and exponent tags are obtained. The result is then formatted into an HTML table, which is then inserted into the final page.

xmlHttp.onreadystatechange = function() {
    if(xmlHttp.readyState == 4) {
        var xmlData = xmlHttp.responseXML;
        var rootNode = xmlData.getElementsByTagName('factors')[0];
        var factorList = rootNode.getElementsByTagName('factor');
        var exponentList = rootNode.getElementsByTagName('exponent');
        var newHTML = "<table border='1'>";
        var num = document.getElementById("inputvalue").value;
        newHTML = newHTML + "<tr><th colspan='2'>Factorization of " + 
                    num + "! </th></tr>";
        newHTML = newHTML + "<tr><th>Factor</th><th>Exponent</th></tr>";
        for ( var i = 0; i < factorList.length; i++) {
            newHTML = newHTML + "<tr>";
            newHTML = newHTML + "<td align='center'>" + 
                          factorList[i].childNodes[0].nodeValue + "</td>";
            newHTML = newHTML + "<td align='center'>" + 
                          exponentList[i].childNodes[0].nodeValue + "</td>";
            newHTML = newHTML + "</tr>";
        }
        newHTML = newHTML + "</table>";
        document.getElementById("result").innerHTML=newHTML;
    }
} 

Part of the HTML is as follows.

<p><span id="result"></span></p>

<br/>
<br/>
<p>
Input: <input type="text" value="20" id="inputvalue" />
</p>
<button onclick="loadData()">Calculate</button>

This shows the span tag that takes the result, the input tag that holds the input value, and the button to set off the interaction. As is typical for AJAX, when the button is clicked the page itself does not change, just part of the page.

Mathematica SOAP Client

Mathematica provides a nice interface for calling SOAP web services with the Web Services Link. This is complemented by webMathematica, which provides functionality for publishing SOAP web services. It is quite convenient to use the Mathematica client tools in order to test the server tools.

This section provides a short description of the Mathematica SOAP client. More details can be found in the Web Services Link user guide.

The first step to using a web service in Mathematica is to install it. This is done with the InstallService function; giving the URL that describes the service, it returns a list of functions that are provided by the service. An example is shown below (note that this service comes from a third party and might stop working at some time in the future).

In[1]:=

Out[1]=

This service returned one function: ValidateZip. This can be used like any other Mathematica function, for example, if the web service provides a description of the service, this is found in the normal way for Mathematica.

In[2]:=

The description tells you how to call the service. This looks up the ZIP Code of the main Wolfram Research office and returns information such as the state and location.

In[3]:=

Out[3]=

This looks up a Zip Code that does not exist.

In[3]:=

Out[3]=

Note that when you call this function, you are making a call to the server that prepares and returns the result.

A key element of SOAP web services is the use of the web services description language, typically referred to as the WSDL of the web service. This describes the service in detail so that other applications can automatically build an interface to the service. For a dynamic language, such as Mathematica, this can be done in Mathematica itself.

You can see the WSDL for a service by entering the URL in a web browser (typically these end with WSDL). For the example service here, enter http://www.webservicemart.com/uszip.asmx?WSDL into your browser. You should see something like the following, which is truncated for brevity.

<?xml version="1.0" encoding="utf-8" ?> 
- <wsdl:definitions xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" xmlns:tns="http://webservicemart.com/ws/" xmlns:s="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soap12/" xmlns:http="http://schemas.xmlsoap.org/wsdl/http/" targetNamespace="http://webservicemart.com/ws/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"> 
- <wsdl:types> 
- <s:schema elementFormDefault="qualified" targetNamespace="http://webservicemart.com/ws/"> 
- <s:element name="ValidateZip"> 
- <s:complexType> 
- <s:sequence> 
<s:element minOccurs="0" maxOccurs="1" name="ZipCode" type="s:string" /> 
</s:sequence> 
</s:complexType> 
</s:element> 
- <s:element name="ValidateZipResponse"> 
- <s:complexType> 
- <s:sequence> 
<s:element minOccurs="0" maxOccurs="1" name="ValidateZipResult" type="s:string" /> 
</s:sequence> 
</s:complexType>
.....

Entering the WSDL into a browser is a good way to check that the service is running.

webMathematica SOAP Services

webMathematica supports the publishing of SOAP web services. There are a number of examples that demonstrate this functionality. These are discussed in this section.

Echo Example

The first example is one that simply echos its input. Note that if you had a toolkit for calling SOAP web services you could use that, but it is quite simple to use Mathematica, as is done here.

The source for this example can be found in the webMathematica web application in the directory Examples/WebServices/Echo.m (the full path in Tomcat would be webapps/webMathematica/Examples/WebServices/Echo.m). If your server is configured and running, you can test this example from within Mathematica with the URL http://localhost:8080/webMathematica/Examples/WebServices/Echo.m?wsdl. (You may have some other URL for accessing your server.)

This installs the service, returning a list of Mathematica functions.

In[1]:=

Out[1]=

These functions can now be used in Mathematica.

In[2]:=

In[3]:=

Out[3]=

The source for the web service consists of a Mathematica code file that is placed in the web content area of the webMathematica web application. A minimal version of the code follows. You can see how each function simply returns its input.

BeginPackage["Echo`", {"WebServicesServer`", "XMLSchema`"}]

(* Simple Mathematica data *)
EchoString::usage = "Echos a String."
EchoInteger::usage = "Echos an Integer."
EchoReal::usage = "Echos a Real."

Begin["`Private`"]

EchoString[str_String] := str
ServiceReturn[EchoString] = _String

EchoInteger[int_Integer] := int
ServiceReturn[EchoInteger] = _Integer

EchoReal[real_Real] := real
ServiceReturn[EchoReal] = _Real

End[]

EndPackage[]

This is written as a Mathematica package: the contents are encapsulated with BeginPackage and EndPackage. You can read more about this in the tutorial on setting up packages. However it has a few additional elements: you need to also import the WebServicesServer` and XMLSchema` packages. Also the public functions, which will be published as web services, need to have input patterns that the server accepts, and the return value of the function needs to be given with the ServiceReturn setting. Typical Mathematica functions do not need to specify any type information, because the system is dynamically typed. However, SOAP web services need this information.

Plot Example

This example provides a web service that returns a plot of a function over a range. Note that if you had a toolkit for calling SOAP web services you could use that, but it is quite simple to use Mathematica, as is done here.

The source for this example can be found in the webMathematica web application in the directory Examples/WebServices/Plot.m (the full path in Tomcat would be webapps/webMathematica/Examples/WebServices/Plot.m). If your server is configured and running you can test this example from within Mathematica with the URL http://localhost:8080/webMathematica/Examples/WebServices/Plot.m?wsdl. (You may have some other URL for accessing your server.)

This installs the service and returns a list of Mathematica functions.

In[2]:=

Out[2]=

In[3]:=

This calls PlotString passing the argument Sin[x] as a string and the number 10. The result is a GIF bitmap of the image (returned as a list of bytes). A summary is shown in the following.

In[4]:=

Out[5]//Short=

This turns the bytes into a string, and uses ImportString to display the image that was generated.

In[6]:=

Out[6]=

This shows the implementation of the Plot web service.

BeginPackage["Plot`", {"WebServicesServer`","MSP`", "XMLSchema`"}]

PlotString::usage = "Plots a function of x from 0 to a limit. 
  The function is passed in as an InputForm string and 
  the result is a GIF."

PlotExpression::usage = "Plots a function of x from 0 to a 
  limit. The function is passed in as an InputForm expression 
  and the result is a Graphics expression."

Begin["`Private`"]

PlotString[expr_String, limit_Integer] :=
    Module[{e, result},
        e = MSPToExpression[expr];
        result = With[{var = Symbol["x"]}, Plot[e, {var, 0, limit}]];
        SchemaBase64Binary[ToCharacterCode[ExportString[result, "GIF"]]]
    ]
ServiceReturn[PlotString] = _SchemaBase64Binary

End[]

EndPackage[]

One important detail here is how the implementation uses MSPToExpression to convert the string input into a Mathematica expression. This is necessary for security. If the service is invoked with an input that does not pass the security test, a failure results.

In[7]:=

Out[7]=

One of the advantages of using SOAP is that it has standards for how error conditions should be handled and reported. Any client toolkit can support these errors, which takes the burden away from the developer.

Excel Example

This example demonstrates how to call a Mathematica web service from Excel. The Mathematica code is Integrator.m, and the source can be found in the webMathematica web application in the directory Examples/WebServices/Plot.m (the full path in Tomcat would be webapps/webMathematica/Examples/WebServices/Plot.m). The Excel spreadsheet file, WebServicesExample.xls, can be found in the Extras directory in your webMathematica distribution.

This shows the implementation of the Integrator web service.

BeginPackage["IntegrateService`", {"WebServicesServer`", "MSP`"}]

IntegrateString::usage = "Integrates an equation with respect to x.  
  The function is passed in as an InputForm string and the result is an 
  InputForm string."

Begin["`Private`"]

IntegrateString[str_String] :=
    Module[{e, result},
        e = MSPToExpression[str];
        result = Integrate[e, Symbol["x"]];
        ToString[result, InputForm]
    ]

ServiceReturn[IntegrateString] = _String

End[]

EndPackage[]

While it would be quite straightforward to call this with the Mathematica web services client, it is more interesting to use the Excel spreadsheet. The example, named WebServicesExample.xls, must be loaded into Excel. You need to allow macros to run, and then it looks something as shown below.

You enter an input into the Expression field, click the Integrate button and the result of the integration comes into the Answer field. If this does not work, you should check the URL that is used, which is written in the script that drives the example.

The actual work for this is done in a Visual Basic script that is included with the spreadsheet. While the documentation here does not include a detailed primer on programming in Visual Basic, it reviews the key elements of the script.

A first part of the script creates the SOAP message as shown below.

Private Sub SetSoapMessage(ByRef Connector As SoapConnector, _
                            ByRef Serializer As SoapSerializer)

    Connector.Property("EndPointURL") = _
      "http://localhost:8080/webMathematica/Examples/Webservices/Integrator.m"
    Call Connector.Connect
    Call Connector.BeginMessage
    Serializer.Init Connector.InputStream
    Serializer.startEnvelope , ENC
    Serializer.SoapNamespace "xsi", XSI
    Serializer.SoapNamespace "SOAP-ENC", ENC
    Serializer.SoapNamespace "xsd", XSD
    Serializer.startBody
    Serializer.startElement "IntegrateString", 
        "http://localhost:8080/webMathematica/Examples/Webservices/Integrator.m", 
        , "method"
    Serializer.startElement "str", "", , ""
    Serializer.writeString Cells(5, 2)
    Serializer.endElement
    Serializer.endElement
    Serializer.endBody
    Serializer.endEnvelope
    Connector.EndMessage

End Sub

Note how it does not use the WSDL, but creates the SOAP message directly. It specifies the URL for the example, gives the name of the function, IntegrateString, and the name of the argument of this function, str. The value of the argument is taken from a cell in the spreadsheet.

Another part of the script gets the response from the server. This appears in the following.

Private Sub GetTagValueArray(ByRef node As MSXML2.IXMLDOMNode)
    Dim childnode As MSXML2.IXMLDOMNode
    If node.baseName = "IntegrateStringReturn" Then
        If node.childNodes.Length = 3 Then
            For Each childnode In node.childNodes
                If childnode.baseName = "element" Then
                    Cells(7, 2) = childnode.Text
                End If
            Next
        End If
    End If
    If node.childNodes.Length > 0 Then
        For Each childnode In node.childNodes
            GetTagValueArray childnode
        Next
    End If
    Exit Sub
End Sub

Note how this uses tools to walk down through the resulting XML to find the IntegrateStringReturn node and take the text from its element node. This has the result, which is then inserted into a cell in the spreadsheet.

Many languages have their own tools for working with SOAP web services, some of them work with WSDL and some directly with the SOAP messages. Whatever the toolkit, it should be possible to call to a Mathematica web service served from webMathematica.

Type Specification

SOAP web services need to specify the type for the input and the type of the result. This is to be constrasted with normal Mathematica usage that does not use any type information, because the system is dynamically typed.

Type information is given by marking up the code with patterns such as shown below for input and for output.

MyFunction[ arg1_spec21, arg2_spec2, ...]

ServiceReturn[MyFunction] = _spec

The specification for an array of a type is given with a list pattern notation. The following example takes an array of integers as an input and returns a list of real numbers.

MyFunction[ arg:{___Integer}]

ServiceReturn[MyFunction] = {___Real}

The ways that types can be specified are summarized in the following table.

Web services data types.

More description of the data types now follows. Many of these will use the Echo.m example. The source for this example can be found in the webMathematica web application in the directory Examples/WebServices/Echo.m (the full path in Tomcat would be webapps/webMathematica/Examples/WebServices/Echo.m). If your server is configured and running, you can test this example from within Mathematica with the URL http://localhost:8080/webMathematica/Examples/WebServices/Echo.m?wsdl. (You may have some other URL for accessing your server.)

Simple Data

Simple data types are the basic building blocks. To use a simple data type such as a string, integer, real, or boolean, you use the appropriate pattern in an argument to a function or inside ServiceReturn. The simple data types are summarized in the following table.

Simple data types.

An example of the use of a basic data type is shown below.

TestInteger[int_Integer] := MatchQ[ int, _Integer]

In this example, TestInteger takes an integer as a parameter and confirms that it is an integer, so the function will always return True when called as a web service.

The Echo.m web service also provides a good way to experiment with the different data types.

In[1]:=

Out[1]=

In[2]:=

Out[2]=

Date and Time Data

There are a number of data types for representing dates and times. To use a date or time data type you need only to use the pattern that applies to the data type when you define a function or ServiceReturn. The data types are summarized in the following table.

Date and time data types.

An example of the use of a date and time data type is shown below.

TestDateTime[dt_SchemaDateTime] := MatchQ[ dt, SchemaDateTime[{y_,m_,d_,h_,m_,s_}]]

In this example, TestDateTime takes a SchemaDateTime expression and confirms that it is a valid date time, so the function will always return True when called as a web service.

The Echo.m web service also provides a good way to experiment with the different data types.

In[1]:=

Out[1]=

In[2]:=

Out[2]=

Binary Data

SchemaBase64Binary is used to transmit binary data, for example from a bitmap image.

Binary data type.

An example of the use of a date and time data type is shown below.

TestBinaryData[b_SchemaBase64Binary] := 
    MatchQ[ b, SchemaBase64Binary[{__Integer}]]

In this example, TestBinaryData takes a SchemaBase64Binary expression and confirms that it is valid binary data, so the function will always return True when called as a web service.

The Echo.m web service also provides a good way to experiment with the different data types.

In[1]:=

Out[1]=

In[3]:=

Out[3]=

SchemaExpr

The SchemaExpr data type is used for transmitting general Mathematica expressions. This might be more useful when it is Mathematica at both ends of the service. However, in principle any system could create the Mathematica expression XML that is used to transport the SOAP message.

Expression data type.

You can get an idea of the structure of the expression XML by exporting a general Mathematica expression. An example is shown below.

In[1]:=

Out[1]=

It would be possible to work with this XML format in a system that was not Mathematica.

When Mathematica converts a SchemaExpr back into a Mathematica expression, the current security system is used to validate the result. If it fails then a security exception results.

So that SchemaExpr can be used to transmit Mathematica expressions without their evaluating, it has the attribute HoldAllComplete.

The Echo.m web service also provides a good way to experiment with the different data types.

In[1]:=

Out[1]=

In[2]:=

Out[2]=

SchemaMathML

The SchemaMathML data type is used for transmitting MathML. Generally this is not so useful when Mathematica is the client. Typically it would be more useful with some other client.

MathML data type.

Mathematica has many functions for working with MathML. For example, the following generates a string of MathML.

In[1]:=

Out[1]=

Alternatively, you can represent the MathML as symbolic XML, as shown in below.

In[2]:=

Out[2]=

The Echo.m web service also provides a good way to experiment with the different data types.

In[3]:=

Out[3]=

Here, you can see how the MathML, formatted in symbolic XML, is returned back.

In[4]:=

Out[4]=

Arrays

Arrays are supported with a convenient and easy syntax, specified by using the type in a repeated pattern.

Array data type.

Examples of the use of array data types are shown below.

TestIntegerArray[list:{___Integer}] := MatchQ[ list, {___Integer}]

TestRealArray[list:{___Real}] := MatchQ[ list, {___Real}]

In these examples, TestIntegerArray takes an array of integers and confirms that it is a valid array, so the function will always return True when called as a web service. TestRealArray does an equivalent test for an array of reals.

The Echo.m web service also provides a good way to experiment with the different data types.

In[1]:=

Out[1]=

In[2]:=

Out[2]=

Errors and Exceptions

There are a number of different types of errors that can result from a web service call. These can fall into the category of HTTP errors, SOAP errors, and function errors.

HTTP errors result from the actual SOAP call, for example the service cannot be found or the server returned some other type of HTTP error. In this case, a SOAP message does not come back as the result, but the client should still handle it.

In the following example, a non-existent web service is called and an HTTP 500 error is returned.

In[1]:=

Out[1]=

A similar error would happen if a call was made to a server that was not operating.

If the HTTP transport works correctly, the SOAP message itself can contain a number of errors. An example of this is the security error. This can be demonstrated with the Echo.m web service. First, the service is installed.

In[2]:=

Out[2]=

Now EchoExpression is called, but it will pass something that could be a security error. This is detected by the server and a SOAP error is returned.

In[3]:=

Out[3]=

Finally, the function itself can return an error. This is shown with the ErrorHandling.m service. First, the service is installed.

In[2]:=

Out[2]=

This shows the day of the week for a valid date.

In[3]:=

Out[3]=

But if an invalid date is used, an error results.

In[4]:=

Out[4]=

The implementation of the function shows that it first checks to see if the input is a valid date. If it is not, it throws a ServiceException, and this causes the error to be returned.

DayOfTheWeek[year_Integer, month_Integer, day_Integer] :=
    Module[{dayOfWeek},
        If[!DateQ[{year, month, day}],
            Throw[ServiceException["Invalid date: " <> ToString[{year, month, day}]]]
        ];
        ToString[ DayOfWeek[{year, month, day}]]
    ]

Security

The discussion here will focus on security implications of web services for the server. Before going further you should be sure to be familar with the section on webMathematica security.

Web services introduce two security issues: processing the SchemaExpr data type and interpreting strings.

If you use SchemaExpr to transmit input to a webMathematica web service, its input will be validated by the security system before it reaches the code that implements the web service.

The Echo.m web service also provides a good way to experiment with the different data types.

In[1]:=

Out[1]=

This makes a call to the web service that echos an expression, but the argument is something that might compromise the security. An error results.

In[2]:=

Out[2]=

This is the code for the EchoExpression web service. Note that it does not have to make any special check for security; that is done as part of the transmission of SchemaExpr.

EchoExpression[expr_SchemaExpr] := expr
ServiceReturn[EchoExpression] = _SchemaExpr

Another security issue arises from transmitting a string to the service. The string represents Mathematica input and it needs to be interpreted. The rule is that you should use MSPToExpression. This will use the webMathematica security system.

An example from the PlotString web service is shown below. Note how the command MSPToExpression is used.

PlotString[expr_String, limit_Integer] :=
    Module[{e, result},
        e = MSPToExpression[expr];
        result = With[{var = Symbol["x"]}, Plot[e, {var, 0, limit}]];
        SchemaBase64Binary[ToCharacterCode[ExportString[result, "GIF"]]]
    ]
ServiceReturn[PlotString] = _SchemaBase64Binary

This installs the Plot.m web service.

In[3]:=

Out[3]=

This triggers a security error (arising from MSPToExpression).

In[4]:=

Out[4]=

A summary of security issues appears below.