This is documentation for Mathematica 6, which was
based on an earlier version of the Wolfram Language.
View current documentation (Version 11.1)

Transforming XML

Introduction

Mathematica is uniquely suited for processing symbolic expressions because of its powerful pattern-matching abilities and large collection of built-in structural manipulation functions. This tutorial provides a few examples to illustrate the use of Mathematica for processing XML data.
When you import an arbitrary XML document into Mathematica, it is automatically converted into a symbolic XML expression. Symbolic XML is the format used for representing XML documents in Mathematica syntax. The conversion to symbolic XML translates the XML document into a Mathematica expression, while preserving its structure. The advantage of converting XML data into symbolic XML is that you can directly manipulate symbolic XML using any of Mathematica's built-in functions.
Convert an XML string into symbolic XML.
In[1]:=
Click for copyable input
Out[1]=
Use a simple transformation rule to remove the unwanted "red" element from the list.
In[2]:=
Click for copyable input
Out[2]=
Use ExportString to convert symbolic XML into native XML syntax, which was designed to be easy to read.
In[3]:=
Click for copyable input
Out[3]=

Visualizing the XML Tree

Many XML tools display an XML document as a collapsible tree, where the nodes correspond to the elements of the document. This example shows how to produce a similar visualization using cell grouping in a Mathematica notebook.
You will do this by recursively traversing the symbolic XML expression and creating a CellGroupData expression that contains cells for each of XMLElement object's attributes and children. Each nested CellGroupData expression will be indented from the previous one. You start with the function to process an XMLElement object.
In[8]:=
Click for copyable input
Use the integer m for indentation. When you map XMLNote onto the XMLElement object's children, you pass a larger value for m, increasing the indentation for the child elements.
A CellGroupData expression contains a list of cells. In the definition, you have only created a cell for the XMLElement x. However, you have then mapped XMLNote onto the attribute list. Since this returns a list, you need to use Apply[ Sequence] to the result in order to merge that list into the CellGroupData expression's list of cells. You then do the same thing to the children of the XMLElement.
However, you have not yet defined XMLNote to work on attributes. The attributes of an XMLElement object are stored in symbolic XML as rules. In most cases, the rule contains two strings: the key and the value. However, when namespaces are involved, the first element of the rule may be a list containing two strings: the namespace and the key. You will need to make two definitions to handle the attributes.
In[9]:=
Click for copyable input
In[10]:=
Click for copyable input
We will need one more definition in order to process simple symbolic XML expressions. The text nodes in an XML document are stored simply as String objects in symbolic XML. Thus, you need a definition that handles String objects.
In[11]:=
Click for copyable input
Construct a simple notebook to visualize a basic XML document.
In[12]:=
Click for copyable input
Out[12]=
In[13]:=
Click for copyable input
Out[13]=
Since the default value of the option "IncludeEmbeddedObjects" is None, you did not alter comments, processing instructions, or anything else that would be stored in an XMLObject. Adding definitions for these is not difficult and would be a good exercise in processing symbolic XML.

Manipulating XML Data

XML applications are used for more than just document layout. XML is also an excellent format for storing structured data. Many commercial database vendors are now adding XML support to their products, allowing you to work with databases using XML as an intermediate format.
Mathematica's symbolic pattern-matching capabilities make it an ideal tool for extracting and manipulating information from XML documents. To illustrate this, manipulate an XML file containing data on major league baseball players.
Import this file into Mathematica as symbolic XML.
In[14]:=
Click for copyable input
Each player's information is stored in a PlayerRecord element. Extract this with Cases.
In[15]:=
Click for copyable input
In[16]:=
Click for copyable input
Out[16]=
The XML document contains records for 294 players. Inside each PlayerRecord, there is a TEAM element which specifies a player's team. By passing a slightly more sophisticated pattern to Cases, you can extract a list of all players on the Yankees team.
In[17]:=
Click for copyable input
In[18]:=
Click for copyable input
Out[18]=
The variable yankees now contains a list of symbolic XML expressions for all the Yankees players. Look at the first element of yankees.
In[19]:=
Click for copyable input
Out[19]=
The player's name is stored in the PLAYER element of each PlayerRecord element. Extract the name from one PlayerRecord element.
In[20]:=
Click for copyable input
Out[20]=
Use Map to extract all the names from yankees.
In[21]:=
Click for copyable input
Out[21]=
Alternatively, use Cases on yankees with an appropriate pattern.
In[22]:=
Click for copyable input
Out[22]=
Symbolic XML is a general-purpose format for expressing arbitrary XML data. In some cases, you may find it more useful to convert symbolic XML into a different type of Mathematica expression. This type of conversion is easy to do using pattern matching.
Import an XML file containing data about baseball pitchers and translate the symbolic XML into a list of rules.
In[23]:=
Click for copyable input
In[24]:=
Click for copyable input
In[25]:=
Click for copyable input
In[26]:=
Click for copyable input
In[27]:=
Click for copyable input
In[28]:=
Click for copyable input
In[29]:=
Click for copyable input
Out[29]=
All the information about the player is stored in a list of Mathematica rules with Pitcher as the head.
In addition to transforming the data into a different expression syntax, you can also modify the data and leave the overall expression in symbolic XML. This way you can alter our data, but still export it to an XML file for use with other applications. As an example, you will work with the salaries of American League hitters.
Delete any PlayerRecord entries where the salary is not available.
In[30]:=
Click for copyable input
Create a function to extract name-salary pairs. Then sort these pairs by salary and look at the top 10.
In[31]:=
Click for copyable input
In[32]:=
Click for copyable input
Out[32]//TableForm=
To illustrate changing data in symbolic XML, create a function that doubles players' salaries.
In[33]:=
Click for copyable input
In[34]:=
Click for copyable input
In[35]:=
Click for copyable input
Out[35]//TableForm=

Visualizing XML Data

Creating a 3D Graphic from an XML File

You can illustrate how to use Mathematica programming and symbolic XML to visualize data in XML format. The molecule description markup language (MoDL) is an XML application that describes molecules. For details, see http://www.oasis-open.org/cover/modl.html. In this example, you convert a MoDL description of the methane molecule into a Mathematica 3D graphic.
The MoDL file that contains the description of the methane molecule.
In[2]:=
Click for copyable input
Out[2]=
Import the file into Mathematica in symbolic XML.
In[37]:=
Click for copyable input
Define a function called MoDLToGraphics3D that turns the symbolic XML into a Graphics3D expression. This function relies on a number of auxiliary functions that are defined in Implementation Details.
In[39]:=
Click for copyable input
Apply this function to the original symbolic XML to generate a 3D graphic representing the methane molecule.

Implementation Details

The original MoDL file contains a head and a body. In the head, a number of definitions are made, which are used throughout the body. You have extracted these definitions into the variable defs. You then map the function ProcessDefinition across the list of definitions. ProcessDefinition constructs a Mathematica expression out of a definition and stores it in the variable moldef, which is dynamically scoped inside of MoDLToGraphics3D.
A DEFINE element in the head typically defines either an atom or a molecule. First, consider an atom definition.
In[41]:=
Click for copyable input
Out[41]=
The DEFINE element essentially associates a unique key (in this case C) to an atom element. The atom element specifies its color and radius. You will turn this into a Mathematica expression of the form Atom[radius, color] and then store it in moldef[name], where name is the key specified in the DEFINE element.
In[42]:=
Click for copyable input
In this case, a is the entire sequence of attributes of the atom element. GetRad and GetColor are functions you will define later that extract the radius and color from this sequence. For now, assume that GetRad returns a number and that GetColor returns an RGBColor. You now need to process definitions of molecule elements. Like the atom definitions, molecules are given a unique key in the name attribute of the DEFINE element. The molecule element then contains atom elements and bond elements.
In[43]:=
Click for copyable input
Out[43]=
The atom elements contain three attributes: type, id, and position. The type attribute references the key from previous atom definitions. The id attribute is a unique key for this instance of the type of atom defined. In other words, the atom elements inside a molecule element represent a distinct atom of some previously defined type.
The molecule element also contains bond elements. These have two attributes: atom1 and atom2. These reference the id of the atom elements in that molecule expression.
When you call ProcessDefinition on a molecule definition, you will want to store a list of the atoms and bonds in moldef.
In[44]:=
Click for copyable input
In[45]:=
Click for copyable input
In[46]:=
Click for copyable input
In the definition of ProcessDefinition, subdef is the list of atoms and bonds. You map ProcessSubdef onto this list. By assigning to moldef[name] a list of the result of ProcessSubdef on each atom and bond element in the molecule. When ProcessSubdef is called on an atom, it extracts that atom's type from moldef, appends the position to that expression, and stores the results under moldef[id]. When ProcessSubdef is called on a bond, it simply returns a Bond expression containing the positions of the two atoms it references.
You need an auxiliary function before you define GetRad, GetColor, and GetPos. Since positions are written as space-separated lists of numbers in MoDL, you first write a function that turns this string into a Mathematica list.
In[47]:=
Click for copyable input
The functions GetPos, GetRad, and GetColor should take in a sequence of attributes of any length and create a list from the position attribute. In symbolic XML, attributes are stored as Mathematica rules. Both GetPos and GetColor will need to use MolStringListToList. GetRad needs only to convert a string to a number.
In[48]:=
Click for copyable input
In[49]:=
Click for copyable input
In[50]:=
Click for copyable input
In[51]:=
Click for copyable input
Here is the definition of MoDLToGraphics3D again for reference.
In[52]:=
Click for copyable input
Block scopes the variables defs, body, moldef, themols, and theatoms. moldef was already discussed, and defs simply contains a list of the DEFINE elements. The function body just contains the body element of the symbolic XML expression.
After ProcessDefinition is mapped to defs, themols is defined. Molecules in the body have a type attribute, which references the key of the molecule type defined in the head. The Cases statement then matches the molecules in the body and returns that molecule's type definition in moldef.
In[53]:=
Click for copyable input
In[54]:=
Click for copyable input
Out[54]=
In this example, you have only one molecule in the body. If more molecules existed, the lists of Atom and Bond expressions would be merged with Flatten. The Graphics3D expression is simply made by drawing each Atom and Bond. Also, the body may contain other atoms as well. The definition of theatoms simply matches these elements, reads their type from moldef, and appends their positions. If more molecules existed, theatoms would contain a list of more atoms to be drawn.
The last line of MoDLToGraphics3D joins the Atom and Bond expressions in themols with the Atom expressions in theatoms. It then maps MolToGraphics onto this list. MolToGraphics is simply a function that returns a sphere for Atom expressions and a line for Bond expressions. Of course, you also need to define MolToGraphics.
In[55]:=
Click for copyable input
In[56]:=
Click for copyable input
The result is a 3D graphic of methane or any other molecule you have defined in MoDL.

Comparing XSLT and Mathematica

In many situations, you need to transform a document from one XML format into another; one popular technique used for this purpose is XSLT transformations. Mathematica's pattern-matching and transformation abilities allow you to do similar transformations by importing the original document and then manipulating the symbolic XML. This section gives examples of some basic XSLT transformations and shows how to do the equivalent transformations in Mathematica.

A Simple Template

In this example, XML dialect uses the code tag to enclose program code. Typically, this is displayed in a monospace font. If you were to convert such a document to XHTML, you would probably want to use the pre tag for code.

<xsl:template match="code">     <pre class="code">         <xsl:value-of select="."/>     </pre> </xsl:template>

In Mathematica, you can create a function to do the same.
In[58]:=
Click for copyable input
In[59]:=
Click for copyable input

Inserting Attribute Values

Now consider an XML application that uses the termdef element to indicate the definition of a new term. You might want to anchor the definition with an element named a so that you can link directly to that location in the document. Assuming you have templates to handle whatever string formatting is inside the termdef element, you can use the following XSLT.

<xsl:template match="termdef">     <span class="termdef">         <a name="{@id}">[Definition:]&nbsp;&nbsp;</a>         <xsl:apply-templates/>     </span> </xsl:template>

Notice that the name attribute in the resultant XHTML gets the value of the id attribute of the original termdef element.
In Mathematica, you can do the following.
In[60]:=
Click for copyable input

Using Predicates

Consider a more complicated example, which will use XPath predicates. Assume you would like to match a note element, but only if it either has a role attribute set to example or if it contains an eg element as a child. Look at an XSLT template and then explain what it does.

<xsl:template match="note[@role='example' or child::eg]">     <div class="exampleOuter">         <div class="exampleHeader">Example</div>         <xsl:if test="*[1][self::p]">             <div class="exampleWrapper">                 <xsl:apply-templates select="*[1]"/>             </div>         </xsl:if>         <div class="exampleInner">             <xsl:apply-templates select="eg"/>         </div>         <xsl:if test="*[position()>1 and self::p]">             <div class="exampleWrapper">                 <xsl:apply-templates                     select="*[position>1 and self::p]"/>             </div>         </xsl:if>     </div> </xsl:template>

The first xsl:if element checks to see if the first child element is a p element. If it is, then xsl:apply-templates is called on that child. This is similar to calling Map across the results of Cases. In the second xsl:if element, you check if there are p child elements beyond the first child. If so, xsl:apply-templates is called on those. Here is the corresponding Mathematica code.
In[61]:=
Click for copyable input
In[62]:=
Click for copyable input
Out[62]=

Traversing Upward

To select an ancestor or sibling, you have to realize that an XML document is just a stream of characters that follows a grammar. Tools for manipulating XML documents treat XML according to some model. In the case of XSLT (and its path-selection language, XPath), this model is that of a tree. Since Mathematica is a list-based language, it treats XML as nested expression lists.
While these two models are similar, they have important differences. Most notably, in nested lists you do not inherently have any concept of the containing list. Technically, any transformation that can be done with axis types like ancestor can also be done without them. However, it is often convenient to traverse up the XML document.
Next is an example and a discussion of how to implement the same behavior in Mathematica. Consider the following XML document.
In[63]:=
Click for copyable input
Out[63]=
Assume you simply want to have a template that matches bibref elements and replaces them with the text inside of the corresponding bibl element. In XSLT, you would write the following template.

<xsl:template match="bibref">     <xsl:param name="ref">         <xsl:value-of select="@ref"/>     </xsl:param>     <xsl:value-of select="/bibliography/bibl[@id = $ref][1]"/> </xsl:template>

The problem with using the same approach in Mathematica is that once you have matched a bibref element, you no longer have any information about the elements containing it. As a remedy, you will instead pass an expression containing the entire symbolic XML expression. Notice that the bibref element in question can be obtained from the following.
In[64]:=
Click for copyable input
Out[64]=
You can pass this expression wrapped in Hold, so that you can easily obtain the bibref element by calling ReleaseHold. You can access ancestors by dropping indices from the Part expression. However, you will need to write a pattern-matching function so that you can match these in definitions of functions.
In[65]:=
Click for copyable input
In[66]:=
Click for copyable input
Out[66]=
The Mathematica transformation then becomes relatively simple.
In[67]:=
Click for copyable input
In[68]:=
Click for copyable input
In[69]:=
Click for copyable input
Out[69]=

Converting a Notebook to HTML

Suppose you need to export a notebook in a specific XML format (one not listed under the FileSave As dialog's Save As Type: submenu). One option would be to export to ExpressionML and then use some external tool (e.g., XSLT rules) to transform to the desired form of XML. But often it is just as easy to perform the manipulation within Mathematica, converting the notebook expression directly into symbolic XML and saving the latter. Anyone with a basic command of Mathematica patterns and programming should be able to do this. Users coming from an XSLT background may even feel a sense of deja vu; since Mathematica expressions are essentially trees, the techniques are much the same.
As an example, re-create an abridged version of the FileSave AsWeb Page (*.html) functionality.
Create an example notebook.
In[70]:=
Click for copyable input
In[71]:=
Click for copyable input
Out[71]=
Define a recursive function, transform, to process the original notebook expression from top to bottom, similar to the templates of XSLT.
Establish a default definition to discard anything not explicitly matched by other patterns.
In[73]:=
Click for copyable input
In[74]:=
Click for copyable input
The definition uses Sequence[] since transform will be applied recursively. The best "null'' result is one that can be dropped in the midst of a list of arguments without disrupting the syntax.
Start with the notebook expression itself.
In[75]:=
Click for copyable input
Note:
  • The argument pattern must be robust enough to accept all variants. (Even though the notebook options are discarded in this conversion, a BlankNullSequence (___) is included to allow for them.)
  • The only thing done with the contents argument is to pass it back to transform .
  • The third argument is always a List. Forgetting this is a common pitfall.
  • Those familiar with HTML will notice that the head element has been dropped.
The same general theme is followed for the remaining definitions.
Discard cell-grouping information, since the HTML has no use for it.
In[76]:=
Click for copyable input
Translate to Mathematica sectional heads their HTML counterparts.
In[77]:=
Click for copyable input
Since the contents of a Mathematica Text style cell can be a simple string or a list wrapped with TextData if the text has substructure of its own, create a definition for both cases.
In[79]:=
Click for copyable input
Simple strings should just be passed on.
In[81]:=
Click for copyable input
Deal with (simple) font changes.
In[82]:=
Click for copyable input
Here is the final product.
In[84]:=
Click for copyable input
Out[84]=
Get output in a more human-readable form by using ExportString.
In[86]:=
Click for copyable input
Out[379]=
Verify that this is well-formed XML.
In[88]:=
Click for copyable input
Out[88]=
The symbolic XML can be exported to a file, suitable for viewing with a web browser.
In[89]:=
Click for copyable input
Out[89]=
An alternative is to apply a list of replacement rules using ReplaceRepeated (//.).
In[90]:=
Click for copyable input
Out[90]=
The two methods produce identical results.
In[91]:=
Click for copyable input
Out[91]=
Here is how the two methods differ.
  • Since the recursion occurs implicitly via ReplaceRepeated, the latter implementation is cleaner in spots. In particular, contrast the handling of Text cells: the TextData rule can be separated from the Cell rule. The same could be accomplished for the recursive function, but at the cost of additional patterns for the various forms that contents might take (for example, _List versus _String and so on). ReplaceRepeated, by acting on all subexpressions, obviates this need.
  • There is no default rule for the second method. Any unhandled parts of the original Mathematica expression will pass through unchanged, probably rendering invalid XML.
Use Clear to remove the definitions of all the symbols.
In[92]:=
Click for copyable input

Verifying Symbolic XML Syntax

You can use XML`SymbolicXMLErrors to find errors in a symbolic XML expression. This function returns a part specification that you can use with functions like Part or Extract to access the problematic part of your symbolic XML expression.
Import an XML file containing data about the American League hitters.
In[95]:=
Click for copyable input
Assume that any player whose salary is "#N/A" is making $1,000,000. Perform the transformation, but make an easily overlooked mistake and use the string "1000000" as the third element, rather than a list containing that string.
In[96]:=
Click for copyable input
Export produces errors when writing modified XML to a file. Use SymbolicXMLErrors to find the problematic expressions.
In[97]:=
Click for copyable input
Out[97]=
XMLElement::name messages are the first indication that something is wrong. The output of SymbolicXMLErrors, however, shows exactly where something went wrong. ALerrors now contains a list of part specifications where the errors occurred. Examine the first error.
In[98]:=
Click for copyable input
Out[410]=
In[99]:=
Click for copyable input
Out[99]=
In[100]:=
Click for copyable input
Out[100]=
This problem is easy enough to fix.
In[101]:=
Click for copyable input
Out[101]=
The rest of the errors are of the same nature.
In[102]:=
Click for copyable input
Out[102]=
Use Map to fix the rest of the errors in the same way.
In[103]:=
Click for copyable input
Out[103]=
Verify that the mistake is fixed using SymbolicXMLErrors again.
In[104]:=
Click for copyable input
Out[104]=