WOLFRAM

Wolfram Language & System Documentation Center

Entropy[data]

gives the base information entropy of the values in data.

Entropy[k,data]

gives the base k information entropy.

Details and Options
Details and Options Details and Options
Examples  
Basic Examples  
Scope  
Options  
SameTest  
Applications  
Properties & Relations  
Possible Issues  
Neat Examples  
See Also
Related Guides
History
Cite this Page

Entropy

Entropy[data]

gives the base information entropy of the values in data.

Entropy[k,data]

gives the base k information entropy.

Details and Options

  • Entropy is a measure of variation (size of information) within the data.
  • The data can have the following additional forms and interpretations:
  • {x1,x2,…}list of numeric or symbolic values » »
    stringlist of characters in a String string »
    WeightedDatalist of values with weights »
    TimeSeries, TemporalDatavector or array of values (the time stamps ignored) »
  • Entropy[k,{x1, …}] is equivalent to -piLog[k,pi], where pi is the occurrence fraction of the distinct element xi. »
  • The following options can be given:
  • SameTest Automaticfunction to test equality of expressions »
  • Entropy[data,SameTestf] applies f to pairs of data elements to determine whether they should be considered equivalent.

Examples

open all close all

Basic Examples  (3)

Entropy of a list of data:

Entropy of a string:

Entropy of a symbolic list:

One of the seven elements occurs once, another twice, and the third one occurs 4 times:

Scope  (4)

Entropy of a random list of zeros and ones:

Entropy of WeightedData:

Compare to the entropy of the data without weights:

Calculate the entropy of a TimeSeries:

The entropy depends only on the values:

Specify the base:

Default base:

Options  (1)

SameTest  (1)

Specify the comparison test for data elements:

Compare to the default setting:

Applications  (2)

Entropy can be used to compare the levels of variations between datasets:

Calculate the entropy for a path of a TelegraphProcess with rate :

Properties & Relations  (3)

Entropy does not depend on the data values:

Compute the entropy of a list from the definition:

Check:

Entropy of a string is equivalent to the entropy of its characters list:

Characters list:

Possible Issues  (1)

For a symbolic base, the base b log evaluates to a quotient of logarithms:

Neat Examples  (2)

Entropy of a text:

Compare with the Sonnets:

Compare the entropy of the text of the UN Declaration of Human Rights in different languages:

Wolfram Research (2008), Entropy, Wolfram Language function, https://reference.wolfram.com/language/ref/Entropy.html.

Text

Wolfram Research (2008), Entropy, Wolfram Language function, https://reference.wolfram.com/language/ref/Entropy.html.

CMS

Wolfram Language. 2008. "Entropy." Wolfram Language & System Documentation Center. Wolfram Research. https://reference.wolfram.com/language/ref/Entropy.html.

APA

Wolfram Language. (2008). Entropy. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/Entropy.html

BibTeX

@misc{reference.wolfram_2025_entropy, author="Wolfram Research", title="{Entropy}", year="2008", howpublished="\url{https://reference.wolfram.com/language/ref/Entropy.html}", note=[Accessed: 16-November-2025]}

BibLaTeX

@online{reference.wolfram_2025_entropy, organization={Wolfram Research}, title={Entropy}, year={2008}, url={https://reference.wolfram.com/language/ref/Entropy.html}, note=[Accessed: 16-November-2025]}