Tensors

Tensors are mathematical objects that give generalizations of vectors and matrices. In the Wolfram System, a tensor is represented as a set of lists, nested to a certain number of levels. The nesting level is the rank of the tensor.

rank 0scalar
rank 1vector
rank 2matrix
rank krank k tensor

Interpretations of nested lists.

A tensor of rank k is essentially a kdimensional table of values. To be a true rank k tensor, it must be possible to arrange the elements in the table in a kdimensional cuboidal array. There can be no holes or protrusions in the cuboid.

The indices that specify a particular element in the tensor correspond to the coordinates in the cuboid. The dimensions of the tensor correspond to the side lengths of the cuboid.

One simple way that a rank k tensor can arise is in giving a table of values for a function of k variables. In physics, the tensors that occur typically have indices which run over the possible directions in space or spacetime. Notice, however, that there is no builtin notion of covariant and contravariant tensor indices in the Wolfram System: you have to set these up explicitly using metric tensors.

Table[f,{i1,n1},{i2,n2},,{ik,nk}]
create an n1×n2××nk tensor whose elements are the values of f
Array[a,{n1,n2,,nk}]create an n1×n2××nk tensor with elements given by applying a to each set of indices
ArrayQ[t,n]test whether t is a tensor of rank n
Dimensions[t]give a list of the dimensions of a tensor
ArrayDepth[t]find the rank of a tensor
MatrixForm[t]print with the elements of t arranged in a twodimensional array

Functions for creating and testing the structure of tensors.

Here is a 2×3×2 tensor.
In[1]:=
Click for copyable input
Out[1]=
This is another way to produce the same tensor.
In[2]:=
Click for copyable input
Out[2]=
MatrixForm displays the elements of the tensor in a twodimensional array. You can think of the array as being a 2×3 matrix of column vectors.
In[3]:=
Click for copyable input
Out[3]//MatrixForm=
Dimensions gives the dimensions of the tensor.
In[4]:=
Click for copyable input
Out[4]=
Here is the element of the tensor.
In[5]:=
Click for copyable input
Out[5]=
ArrayDepth gives the rank of the tensor.
In[6]:=
Click for copyable input
Out[6]=

The rank of a tensor is equal to the number of indices needed to specify each element. You can pick out subtensors by using a smaller number of indices.

Transpose[t]transpose the first two indices in a tensor
Transpose[t,{p1,p2,}]transpose the indices in a tensor so that the k^(th) becomes the pk^(th)
Tr[t,f]form the generalized trace of the tensor t
Outer[f,t1,t2]form the generalized outer product of the tensors t1 and t2 with "multiplication operator" f
t1.t2form the dot product of t1 and t2 (last index of t1 contracted with first index of t2)
Inner[f,t1,t2,g]form the generalized inner product, with "multiplication operator" f and "addition operator" g

Tensor manipulation operations.

You can think of a rank k tensor as having k "slots" into which you insert indices. Applying Transpose is effectively a way of reordering these slots. If you think of the elements of a tensor as forming a kdimensional cuboid, you can view Transpose as effectively rotating (and possibly reflecting) the cuboid.

In the most general case, Transpose allows you to specify an arbitrary reordering to apply to the indices of a tensor. The function Transpose[T,{p1,p2,,pk}] gives you a new tensor T such that the value of Ti1 i2 ik is given by Tip1 ip2 ipk.

If you originally had an np1×np2××npk tensor, then by applying Transpose, you will get an n1×n2××nk tensor.

Here is a matrix that you can also think of as a 2×3 tensor.
In[7]:=
Click for copyable input
Out[7]=
Applying Transpose gives you a 3×2 tensor. Transpose effectively interchanges the two "slots" for tensor indices.
In[8]:=
Click for copyable input
Out[8]=
The element m[[2,3]] in the original tensor becomes the element m[[3,2]] in the transposed tensor.
In[9]:=
Click for copyable input
Out[9]=
This produces a 2×3×1×2 tensor.
In[10]:=
Click for copyable input
Out[10]=
This transposes the first two levels of t.
In[11]:=
Click for copyable input
Out[11]=
The result is a 3×2×1×2 tensor.
In[12]:=
Click for copyable input
Out[12]=

If you have a tensor that contains lists of the same length at different levels, then you can use Transpose to effectively collapse different levels.

This collapses all three levels, giving a list of the elements on the "main diagonal".
In[13]:=
Click for copyable input
Out[13]=
This collapses only the first two levels.
In[14]:=
Click for copyable input
Out[14]=

You can also use Tr to extract diagonal elements of a tensor.

This forms the ordinary trace of a rank 3 tensor.
In[15]:=
Click for copyable input
Out[15]=
Here is a generalized trace, with elements combined into a list.
In[16]:=
Click for copyable input
Out[16]=
This combines diagonal elements only down to level 2.
In[17]:=
Click for copyable input
Out[17]=

Outer products, and their generalizations, are a way of building higherrank tensors from lowerrank ones. Outer products are also sometimes known as direct, tensor, or Kronecker products.

From a structural point of view, the tensor you get from Outer[f,t,u] has a copy of the structure of u inserted at the "position" of each element in t. The elements in the resulting structure are obtained by combining elements of t and u using the function f.

This gives the "outer f" of two vectors. The result is a matrix.
In[18]:=
Click for copyable input
Out[18]=
If you take the "outer f" of a length 3 vector with a length 2 vector, you get a 3×2 matrix.
In[19]:=
Click for copyable input
Out[19]=
The result of taking the "outer f" of a 2×2 matrix and a length 3 vector is a 2×2×3 tensor.
In[20]:=
Click for copyable input
Out[20]=
Here are the dimensions of the tensor.
In[21]:=
Click for copyable input
Out[21]=

If you take the generalized outer product of an m1×m2××mr tensor and an n1×n2××ns tensor, you get an m1××mr×n1××ns tensor. If the original tensors have ranks r and s, your result will be a rank r+s tensor.

In terms of indices, the result of applying Outer to two tensors Ti1 i2 ir and Uj1 j2 js is the tensor Vi1 i2 irj1 j2 js with elements f[Ti1 i2 ir,Uj1 j2 js].

In doing standard tensor calculations, the most common function f to use in Outer is Times, corresponding to the standard outer product.

Particularly in doing combinatorial calculations, however, it is often convenient to take f to be List. Using Outer, you can then get combinations of all possible elements in one tensor, with all possible elements in the other.

In constructing Outer[f,t,u] you effectively insert a copy of u at every point in t. To form Inner[f,t,u], you effectively combine and collapse the last dimension of t and the first dimension of u. The idea is to take an m1×m2××mr tensor and an n1×n2××ns tensor, with mr=n1, and get an m1×m2××mr-1×n2××ns tensor as the result.

The simplest examples are with vectors. If you apply Inner to two vectors of equal length, you get a scalar. Inner[f,v1,v2,g] gives a generalization of the usual scalar product, with f playing the role of multiplication, and g playing the role of addition.

This gives a generalization of the standard scalar product of two vectors.
In[22]:=
Click for copyable input
Out[22]=
This gives a generalization of a matrix product.
In[23]:=
Click for copyable input
Out[23]=
Here is a 3×2×2 tensor.
In[24]:=
Click for copyable input
Out[24]=
Here is a 2×3×1 tensor.
In[25]:=
Click for copyable input
Out[25]=
This gives a 3×2×3×1 tensor.
In[26]:=
Click for copyable input
Out[26]=
Here are the dimensions of the result.
In[27]:=
Click for copyable input
Out[27]=

You can think of Inner as performing a "contraction" of the last index of one tensor with the first index of another. If you want to perform contractions across other pairs of indices, you can do so by first transposing the appropriate indices into the first or last position, then applying Inner, and then transposing the result back.

In many applications of tensors, you need to insert signs to implement antisymmetry. The function Signature[{i1,i2,}], which gives the signature of a permutation, is often useful for this purpose.

Outer[f,t1,t2,]form a generalized outer product by combining the lowestlevel elements of t1,t2,
Outer[f,t1,t2,,n]treat only sublists at level n as separate elements
Outer[f,t1,t2,,n1,n2,]treat only sublists at level ni in ti as separate elements
Inner[f,t1,t2,g]form a generalized inner product using the lowestlevel elements of t1
Inner[f,t1,t2,g,n]contract index n of the first tensor with the first index of the second tensor

Treating only certain sublists in tensors as separate elements.

Here every single symbol is treated as a separate element.
In[28]:=
Click for copyable input
Out[28]=
But here only sublists at level 1 are treated as separate elements.
In[29]:=
Click for copyable input
Out[29]=
ArrayFlatten[t,r]create a flat rank r tensor from a rank r tensor of rank r tensors
ArrayFlatten[t]flatten a matrix of matrices (equivalent to ArrayFlatten[t,2])

Flattening block tensors.

Here is a block matrix (a matrix of matrices that can be viewed as blocks that fit edge to edge within a larger matrix).
In[30]:=
Click for copyable input
Out[30]//TableForm=
Here is the matrix formed by piecing the blocks together.
In[31]:=
Click for copyable input
Out[31]//TableForm=