URLRead

URLRead[url]

sends a request to a URL and reads back the response, returning it as a response object.

URLRead[assoc]

sends a request to a URL built from an association of components and metadata elements.

URLRead[HTTPRequest[…]]

sends a request specified by a symbolic HTTPRequest object.

URLRead[req,"elem"]

returns only the element elem from the response.

URLRead[req,{"elem1","elem2",…}]

returns an association of the values of the elements elemi.

URLRead[{req1,req2,…},…]

sends the requests reqi in parallel, giving a list of the results obtained.

Details and Options

  • In URLRead[url], url can be a string, a URL object, or a cloud object.
  • The URL can also be specified by giving the following components in an association:
  • "Scheme"URL scheme
    "User"user information
    "Domain"domain name or host
    "Port"port number (as an integer)
    "Path"path list or string
    "Query"query string or list of parameter-value pairs
  • The following additional entries can also be included in the association, as in HTTPRequest:
  • Method"GET"method to use for the HTTP request
    "Body"""contents of message body to be sent
    "ContentType"Automaticcontent-type header for the request
    "Username"""username to use for the request
    "Password"""password to use for the request
    "Cookies"Automaticcookies to send to the server
    "UserAgent"Automaticuser agent string to send
    "Headers"{}list of headers to be sent to the server
  • The setting for "Body" can have the following forms:
  • "string"string to be sent literally
    ByteArray[…]byte array to be sent literally
    {b1,b2,b3,…}list of bytes to be sent literally
    {"par1""val1","par2""val2",…}list of parameters and values to be sent URL encoded
    {"par1"{"val11",…},…}list with multiple values associated with a single parameter
    <|"key1"val1,"key2"val2,…|>alternative form for list of parameters and values
    {"par1"File[…],"par2"File[…],…}files to be sent with multipart encoding
    {"par1"assoc1,…}multipart encoding with detailed specifications for each part
  • Detailed specifications for each part in a multipart encoding are given as an association with the following elements:
  • "Content"the content, as a string, byte array, file object, etc.
    "Name"the name to assign to the part
    "MIMEType"the MIME type to assign to the part
  • The setting "Cookies"->Automatic specifies that cookies from $Cookies for the domain corresponding to url should be sent. "Cookies" can also give an explicit list of associations representing cookies. "Cookies"->None specifies that no cookies should be sent.
  • The default "UserAgent" is typically "Wolfram HTTP Client xxxx", where xxxx is a version number.
  • The setting for "Headers" can be a list of rules or an association.
  • In URLRead[req,elem], the following elements can be used:
  • "Body"body of the response as a string
    "BodyByteArray"raw body of the response as a byte array
    "BodyBytes"raw body of the response as a list of byte values
    "Headers"list of headers received in the response
    "Cookies"list of cookies received in the response
    "ContentType"content type header
    "StatusCode"status code returned by the server
    "StatusCodeDescription"plain text description of status code
    "Stream"stream object to read the response body
    {elem1,elem2,…}an association of the elements elemi
    Automaticsymbolic HTTPResponse object
    Allan association of all elements
  • URLRead supports the following options:
  • AuthenticationAutomaticauthentication information to send
    CharacterEncodingAutomaticcharacter encoding for strings sent
    ConnectionSettingsAutomaticspeed and other settings for the connection
    CookieFunctionAutomaticfunction to apply to each cookie received
    FollowRedirectsTruewhether to follow HTTP redirects
    InteractiveTruewhether to allow interactive authentication dialogs
    TimeConstraintInfinitytime to wait for a response
    VerifySecurityCertificatesTruewhether to verify SSL security certificates
  • The setting CharacterEncoding->"enc" specifies that all strings that appear in the request should be encoded using the encoding enc before being sent to the server.
  • The setting CharacterEncoding->None specifies that no change should be made to strings before sending them to the server.
  • With CharacterEncoding->Automatic, the encoding is taken from the "ContentType" element if this is specified; otherwise, the encoding is "UTF8".
  • TimeConstraint->t allows a maximum of t seconds for connection to the URL and reading from it.
  • TimeConstraint-><|"Connecting"->tc,"Reading"->tr|> specifies separate timeouts for connecting to the URL and reading from it.

Examples

open allclose all

Basic Examples  (2)

Retrieve a URL and get back an HTTP response:

Use a second argument to get different elements of the HTTP response:

More complex requests can use HTTPRequest:

Scope  (2)

Build a URL from components:

Changing query parameters and scheme for a given URL:

To specify parameters with the same name, you can use a list of rules:

Or they can be grouped in a list:

Grouping parameters in a list is useful when working with associations, since they do not allow repeated keys:

If no "Method" is specified, "GET" is used automatically:

The body can be a raw string:

To send an HTTP request where the body is URL encoded, you can just send a list of values. If no "Method" is specified, "POST" is used automatically:

To send files, just use the file object:

To send a more complex file specification, you can use Association:

Content in the Association can be a string, a ByteArray, or a File:

Options  (4)

CookieFunction  (1)

Apply a function to each cookie received when an HTTP response is received:

FollowRedirects  (1)

Specify whether to follow HTTP redirects when retrieving a URL:

TimeConstraint  (1)

Specify the maximum allotted time in seconds to finish downloading data from the server:

VerifySecurityCertificates  (1)

Verify that the SSL certificate used by the server is valid:

Properties & Relations  (3)

File and CloudObject are fully supported:

Use URLDownload to store output in a file:

Use Get for loading Wolfram Language expressions directly over HTTP:

Import the request by using Import or URLExecute:

Possible Issues  (1)

Network requests can go wrong for several reasons; messages are sent to help you debug possible causes, and a Failure object is returned:

The Failure object contains the HTTPRequest object:

Neat Examples  (1)

Import an APIFunction:

Introduced in 2016
 (11.0)