HDF5 (.h5, .hdf5)

Background

    • MIME type: application/x-hdf5
    • HDF data format Version 5.
    • General-purpose format for representing multidimensional datasets and images.
    • Datasets with compound data structures are supported.
    • Used for storage, management, and exchange of scientific data.
    • HDF is an acronym for Hierarchical Data Format.
    • Originally developed by the US National Center for Supercomputing Applications (NCSA).
    • Currently maintained by The HDF Group.
    • Binary file format.
    • Incompatible with HDF Version 4 and earlier.

Import & Export

  • Import["file.h5"] imports an HDF5 file, returning the names of the datasets stored in the file.
  • Import["file.h5",elem] imports the specified element from an HDF5 file.
  • The import format can be specified with Import["file","HDF5"] or Import["file",{"HDF5",elem,}].
  • Export["file.h5",expr] exports a numeric array to HDF5.
  • Export["file.h5",{expr1,},{"Datasets", {"dataset1",}}] creates an HDF5 file, storing the data arrays {expr1,} as separate datasets.
  • Export["file.h5",expr,elem] exports the specified element to an HDF5 file.
  • Export["file.h5",elem1->expr1,,"Rules"] uses rules to specify the elements to be exported.
  • See the reference pages for full general information on Import and Export.
  • ImportString and ExportString support the HDF5 format.

Import Elements

  • General Import elements:
  • "Elements"list of elements and options available in this file
    "Rules"full list of rules for each element and option
    "Options"list of rules for options, properties, and settings
  • Structure elements:
  • "Datasets"names of all datasets
    "Groups"names of all groups
    "StructureGraph"a directed graph showing the structure of the datasets
    {"StructureGraph, groupname}a graph showing the structure under groupname
    "StructureGraphLegend"legend for the structure graph
    "Summary"summary of properties
  • Names of groups and datasets are given as the absolute paths starting with the root group name "/".
  • Import by default uses the "Datasets" element for the HDF5 format.
  • Data representation elements:
  • "Data"all datasets imported as an association
    {"Data",n} or nn^(th) dataset
    {"Data",dataset} or datasetnamed dataset
    {"Data",groupname}an association of all datasets under groupname
    {"Data",groupname,lev}an association of datasets under groupname up to level lev
  • The following basic data types are supported:
  • "Integer8"8-bit integers
    "Integer16"16-bit integers
    "Integer32"32-bit integers
    "Integer64"64-bit integers
    "UnsignedInteger8"8-bit unsigned integers
    "UnsignedInteger16"16-bit unsigned integers
    "UnsignedInteger32"32-bit unsigned integers
    "UnsignedInteger64"64-bit unsigned integers
    "Real32"IEEE singleprecision numbers
    "Real64"IEEE doubleprecision numbers
    "String"string of ASCII characters
  • The following structured data types are supported:
  • "ByteArray"a ByteArray of an arbitrary length
    "Array"an array of any supported data format
    "Enum"an enumeration
    "Compound"a compound dataset consisting of any other data format and other compound datasets
  • Complex numbers are typically stored and imported as compound types.
  • Metadata elements:
  • "Attributes"attributes of all groups and datasets
    "DataEncoding"specifies how each dataset is compressed
    "DataFormat"type used to represent each dataset
    "Dimensions"data dimensions of each dataset
    {"metadata",n}metadata of the n^(th) dataset
    {"metadata",dataset}metadata of the named dataset
  • The following data encodings are supported:
  • Noneno data compression is used
    "Fletcher32"adds the Fletcher checksum
    "GZIP"GZIP compression
    "ScaleOffset"performs a scale and/or offset operation
    "Shuffle"reorders so that consistent byte positions are placed together
    "SZIP"SZIP compression (Import only)
  • A single dataset can have multiple encodings, which will be specified as a list {enc1,enc2,}.
  • Export Elements
  • The following export elements can be given:
  • "Attributes"attributes associated to any object
    "Datasets"datasets and their associated elements
    "Groups"group names and their associated elements
    "NamedDataTypes"named data types and their associated elements
  • With the "Attributes" element, the following expressions can be given:
  • attrattributes associated to the root group "/"
    {path1attr1,}attributes attri associated to the specified pathi
  • Attributes attri should be given in the form "attname"->attval.
  • With the "Groups" element, the following expressions can be given:
  • {"gr1","gr2",}a list of group paths
    {"gr1"grdata1,}list of groups and their associated data
  • Group data grdatai can take the following keys:
  • "Attributes"group attributes
    "HardLinks"hard links to other objects
    "SoftLinks"soft links to other objects
  • Soft and hard links can be specified using "linkname"path.
  • With the "Datasets" element, the following expressions can be given:
  • datastores data under "Dataset1"
    {"name1"data1,}a list of dataset names and their associated data
    {"name1"ds1,}specifies each dataset dsi using a list of rules
  • Datasets dsi can take the following keys:
  • "Attributes"dataset attributes
    "Data"data
    "DataFormat"data type
  • With the "NamedDataTypes" element, the following expressions can be given:
  • {"name1"type1,}a data type
    {"name1"<|"Type"type1,"Attributes"att1|>,}an association specifying a type and its attributes
  • The type specification typei can take the following forms:
  • "simpletype"a simple data type such as "Integer64"
    <|"Class""ByteArray",|>takes "Length" and "Tag" keys
    <|"Class""Array",|>takes "DataFormat" and "Dimensions" keys
    <|"Class""Compound",|>takes a "Structure" key

Options

  • Import and Export options:
  • "ComplexKeys"Automatickeys for complex interpretation and export
  • By default, {"Re","Im"} are used as complex keys. Other settings include:
  • Noneno complex interpretation (import only)
    Automaticuse automatic keys
    {key1,key2}use explicit keys
  • Import option:
  • "TakeElements"Allsubset of elements to import
  • "TakeElements" can take the following values:
  • {elem1,elem2,}list of elements elemi
    {m;;n;;s,...}elements m through n in steps of s
    {opt1val1,}list of suboptions
  • The following suboptions opti are available for taking elements:
  • "Offset"{0,0,}the offset along the dimensions of the dataset
    "Count"Allthe number of block to be imported along each dimension
    "Stride"{1,1,}the step between beginnings of the blocks
    "Block"{1,1,}number of elements in each block
  • Export option:
  • ByteOrdering$ByteOrderingwhat byte ordering to use
    OverwriteTargetTruewhether to overwrite an existing file
    "AppendMode""Extend"how to append to existing objects
  • Using OverwriteTarget->"Append", new objects may be added to an existing file.
  • Possible settings for "AppendMode" include:
  • "Extend"extends existing objects, if possible (default)
    "Overwrite"overwrites exisiting objects
    "Preserve"preserves existing objects

Examples

open all close all

Basic Examples  (1)

Show the datasets stored in a sample file:

In[1]:=
Click for copyable input
Out[1]=

Get the file summary:

In[2]:=
Click for copyable input
Out[2]=

Show the structure of the file:

In[2]:=
Click for copyable input
Out[2]=

Scope  (12)

Import Elements  (22)

Export Elements  (21)

Import Options  (3)

Applications  (1)

Properties & Relations  (1)

Possible Issues  (2)

Introduced in 2004
(5.1)
|
Updated in 2019
(12.0)