--- title: "WordFrequency" language: "en" type: "Symbol" summary: "WordFrequency[text, word] gives the frequency of word in text. WordFrequency[text, {word1, word2, ...}] gives an association of the frequencies of each of the wordi." keywords: - n-grams canonical_url: "https://reference.wolfram.com/language/ref/WordFrequency.html" source: "Wolfram Language Documentation" related_guides: - title: "Text Analysis" link: "https://reference.wolfram.com/language/guide/TextAnalysis.en.md" - title: "Text Normalization" link: "https://reference.wolfram.com/language/guide/TextNormalization.en.md" - title: "Natural Language Processing" link: "https://reference.wolfram.com/language/guide/NaturalLanguageProcessing.en.md" related_workflows: - title: "Analyze the Text on a Webpage" link: "https://reference.wolfram.com/language/workflow/AnalyzeTheTextOnAWebpage.en.md" related_functions: - title: "WordCloud" link: "https://reference.wolfram.com/language/ref/WordCloud.en.md" - title: "WordFrequencyData" link: "https://reference.wolfram.com/language/ref/WordFrequencyData.en.md" - title: "WordCounts" link: "https://reference.wolfram.com/language/ref/WordCounts.en.md" - title: "DictionaryWordQ" link: "https://reference.wolfram.com/language/ref/DictionaryWordQ.en.md" - title: "WordData" link: "https://reference.wolfram.com/language/ref/WordData.en.md" --- # WordFrequency WordFrequency[text, word] gives the frequency of word in text. WordFrequency[text, {word1, word2, …}] gives an association of the frequencies of each of the wordi. ## Details and Options * ``WordFrequency[text, word]`` gives the number of occurrences of ``word`` divided by the total number of words in ``text``. * In ``WordFrequency[text, word]``, ``word`` can be a multiword string containing spaces. For an n-gram string, the result is divided by the total number of n-grams in ``text``. * ``WordFrequency[text, word1 | word2 | …]`` gives the total frequency of all the ``wordi``. * ``WordFrequency[{text1, text2, …}, word]`` gives a list of frequencies in each of the ``texti``. * Possible options include: [`IgnoreCase`](https://reference.wolfram.com/language/ref/IgnoreCase.en.md) [`False`](https://reference.wolfram.com/language/ref/False.en.md) whether to ignore letter casing * ``WordFrequency[text, word, "CaseVariants"]`` gives an association that includes frequencies of all variants of upper and lower case. --- ## Examples (14) ### Basic Examples (2) Find the frequency of the word ``"the"`` in a string: ```wl In[1]:= WordFrequency["the cat in the hat", "the"] Out[1]= 0.4 ``` --- Find the frequencies for a list of words: ```wl In[1]:= WordFrequency["the cat in the hat", {"the", "cat", "hat"}] Out[1]= <|"the" -> 0.4, "cat" -> 0.2, "hat" -> 0.2|> ``` ### Scope (7) #### Basic Uses (4) Find the frequency of ``"had"`` in a given string: ```wl In[1]:= WordFrequency["James while John had had had had had had had had had had had a better effect on the teacher", "had"] Out[1]= 0.55 ``` --- For a list of words, an association of frequencies will be returned: ```wl In[1]:= WordFrequency["James while John had had had had had had had had had had had a better effect on the teacher", { "had", "the", "on"}] Out[1]= <|"had" -> 0.55, "the" -> 0.05, "on" -> 0.05|> ``` --- When using alternatives, the total frequency for the matching pattern will be returned: ```wl In[1]:= WordFrequency["James while John had had had had had had had had had had had a better effect on the teacher", "the" | "on"] Out[1]= 0.1 ``` --- If multiple alternatives are specified, a frequency will be given for each: ```wl In[1]:= WordFrequency["James while John had had had had had had had had had had had a better effect on the teacher", {"the" | "on", "had had" | "on the"}, IgnoreCase -> True] Out[1]= <|"on" | "the" -> 0.1, "had had" | "on the" -> 0.578947|> ``` #### Case Variants (3) A word can have many lower- and uppercase variants: ```wl In[1]:= WordFrequency["That that is, is that THAT is. Not is not. Is that it? It is", "that", "CaseVariants"] Out[1]= <|"That" -> 0.0666667, "that" -> 0.2, "THAT" -> 0.0666667|> ``` --- For multiword inputs, case variants will be returned for each word present: ```wl In[1]:= WordFrequency["That that is, is that THAT is. Not is not. Is that it? It is", {"that", "it"}, "CaseVariants"] Out[1]= <|"That" -> 0.0666667, "that" -> 0.2, "THAT" -> 0.0666667, "it" -> 0.0666667, "It" -> 0.0666667|> ``` Alternatively, an explicit list of case-sensitive words can be provided: ```wl In[2]:= WordFrequency["That that is, is that THAT is. Not is not. Is that it? It is", {"that", "That", "THAT", "it", "It"}] Out[2]= <|"that" -> 0.2, "That" -> 0.0666667, "THAT" -> 0.0666667, "it" -> 0.0666667, "It" -> 0.0666667|> ``` --- Multiple words with ``"CaseVariants"`` will be returned as lists: ```wl In[1]:= WordFrequency["That that is, is that THAT is. Not is not. Is that it? It is", {"that" | "not", "is is" | "that that"}, "CaseVariants"] Out[1]= <|"That" -> 0.0666667, "that" -> 0.2, "THAT" -> 0.0666667, "Not" -> 0.0666667, "not" -> 0.0666667, {"That", "that"} -> 0.0714286, {"is", "is"} -> 0.0714286, {"that", "THAT"} -> 0.0714286|> ``` ### Options (1) #### IgnoreCase (1) By default, frequencies account for only those words that match the case of the specified word: ```wl In[1]:= WordFrequency["Buffalo from Buffalo, which buffalo from Buffalo bully, themselves bully buffalo from Buffalo.", "Buffalo"] Out[1]= 0.307692 ``` With ``IgnoreCase -> True``, all variations in case will be counted together: ```wl In[2]:= WordFrequency["Buffalo from Buffalo, which buffalo from Buffalo bully, themselves bully buffalo from Buffalo.", "Buffalo", IgnoreCase -> True] Out[2]= 0.461538 ``` ### Possible Issues (2) With ``IgnoreCase -> True``, the resulting association will omit keys for case-variant duplicates: ```wl In[1]:= WordFrequency["Sentences should Be split by Words, and words may be split by letters", {"words", "be", "Be", "BE"}, IgnoreCase -> True] Out[1]= <|"words" -> 0.153846, "be" -> 0.153846|> ``` --- Words that do not appear within the given text will have a frequency of zero: ```wl In[1]:= WordFrequency[ExampleData[{"Text", "DeclarationOfIndependence"}], "beltalowda"] Out[1]= 0. ``` ### Neat Examples (2) The frequency of articles in a sample of *Alice in Wonderland* : ```wl In[1]:= WordFrequency[ExampleData[{"Text", "AliceInWonderland"}], {"a", "an", "the"}, "CaseVariants"] Out[1]= <|"a" -> 0.0275513, "A" -> 0.000825508, "an" -> 0.00165102, "An" -> 0.000206377, "THE" -> 0.00072232, "the" -> 0.059127, "The" -> 0.00515943|> In[2]:= WordFrequency[ExampleData[{"Text", "AliceInWonderland"}], {"a", "an", "the"}, IgnoreCase -> True] Out[2]= <|"a" -> 0.0283768, "an" -> 0.00185739, "the" -> 0.0650088|> ``` Create a word cloud using the frequencies for the names of the characters in this book: ```wl In[3]:= WordCloud[WordFrequency[ExampleData[{"Text", "AliceInWonderland"}], {"Alice", "Queen", "Rabbit", "Duchess", "King", "Mouse", "Hatter", "Cat", "Dodo", "Caterpillar"}, IgnoreCase -> True]] Out[3]= [image] ``` --- The frequency of the word "mission" in the Wikipedia article "Moon": ```wl In[1]:= WordFrequency[WikipediaData["Moon"], "mission", IgnoreCase -> True] Out[1]= 0.0026933 ``` ## See Also * [`WordCloud`](https://reference.wolfram.com/language/ref/WordCloud.en.md) * [`WordFrequencyData`](https://reference.wolfram.com/language/ref/WordFrequencyData.en.md) * [`WordCounts`](https://reference.wolfram.com/language/ref/WordCounts.en.md) * [`DictionaryWordQ`](https://reference.wolfram.com/language/ref/DictionaryWordQ.en.md) * [`WordData`](https://reference.wolfram.com/language/ref/WordData.en.md) ## Related Guides * [Text Analysis](https://reference.wolfram.com/language/guide/TextAnalysis.en.md) * [Text Normalization](https://reference.wolfram.com/language/guide/TextNormalization.en.md) * [Natural Language Processing](https://reference.wolfram.com/language/guide/NaturalLanguageProcessing.en.md) ## Related Workflows * [Analyze the Text on a Webpage](https://reference.wolfram.com/language/workflow/AnalyzeTheTextOnAWebpage.en.md) ## History * [Introduced in 2016 (10.4)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn104.en.md)