HTTPCLIENT PACKAGE TUTORIAL

Introduction

The HTTPClient package lets you work with HTTP servers. It lets you send and receive data with remote servers, as well as providing a means of examining the data received from the server. It is useful for downloading remote data to be analyzed in Mathematica and for interacting with different web APIs.

The main function used for the HTTPClient package is URLFetch, which is used to download information from a remote HTTP server.

Basic Usage

You can easily download the contents of a web page using URLFetch.

data = URLFetch["www.wolfram.com"];
StringTake[data, 300]
Out[1]=

Content from the site can also be retrieved as a list of bytes.

URLFetch["www.wolfram.com", "ContentData"]
Out[2]=

Header data can be viewed.

URLFetch["www.wolfram.com", "Headers"]
Out[3]=

Multiple elements can be returned at once.

URLFetch["www.wolfram.com", {"Headers", "Cookies", "StatusCode"}]
Out[4]=

By default, cookies are handled automatically by HTTPClient, and they can be inspected by using $HTTPCookies.

$HTTPCookies
Out[5]=

Sending Data

Parameters can be passed to a server by specifying the option to URLFetch. For example, you can view the HTML returned by searching http://reference.wolfram.com for the Map function.

http = URLFetch["reference.wolfram.com/search.html", "Parameters"->{"query"->"Map"}];
StringTake[http, {20000, 20500}]
Out[6]=

By default, URLFetch will pass data using the HTTP GET method; if another method is required, then the option must be specified.

The option allows you to easily interact with many public web APIs. Here is an example.

url = "http://exampledata.wolfram.com/httpget.php";
name = "Joe";
age = "30";
URLFetch[url, "Parameters"→ {"name"→name, "age"→age}]
Out[10]=

The script is designed to print out information about a person, given their name and age. You can see the parameters passed to the server in the output of the script.

Streams

HTTPClient registers an input stream method, which can be used with several of Mathematica's built-in functions for reading from files.

You can use HTTP streams with any Mathematica function that supports InputStream methods. To use HTTP streams, all URLs must begin with either "http://" or "https://".

ReadList["http://wolframalpha.com", Character, 25]
Out[11]=

Options can be passed to streams in the following way.

strm = OpenRead["http://wolfram.com", Method→{"HTTP", "UserAgent"→"Generic Web Browser"}]
Out[12]=

When you are finished with an InputStream, it should be closed.

Close[strm]
Out[13]=

The Get function also supports InputStream methods. This is useful for evaluating Mathematica expressions over HTTP. Consider the following package for the Collatz conjecture.

URLFetch["http://exampledata.wolfram.com/Collatz.m"]
Out[14]=

Instead of just reading the content with URLFetch, the package can be loaded directly into Mathematica.

Get["http://exampledata.wolfram.com/Collatz.m"]
Collatz[11]
Out[16]=

When a URL is passed directly to functions such as Get without first calling OpenRead, it is not necessary to close the stream with the Close function: this will be done automatically.

New to Mathematica? Find your learning path »
Have a question? Ask support »