Natural Language Processing

Natural language processing deals with understanding text and spoken words as a human would. It is a fundamental component of many human/machine interactions (vocal assistants, dictation software, voice-operated systems, ...), text processing, analysis (suggestions, keyword spotting, translation, ...) and much more. The Wolfram Language natural language processing functionality is a combination of rule-based and machine learning language models, including LLMs. It builds on top of advanced text mining and string manipulation capabilities and is integrated with a large visualization suite and extensive built-in linguistic data.

Text Acquisition

Import import text from files or the web

"Text", "PDF", "HTML", "CSV", pick out plaintext, table data, etc.

TextRecognize  ▪  ExampleData  ▪  WikipediaData

LLMSynthesize generate text from a prompt using an LLM

Text Mining

TextSearch search an index or directory, returning a list of documents

Find, FindList search files for records containing particular strings

StringTake  ▪  StringReplace  ▪  StringCases  ▪  RegularExpression  ▪  ...

Text Normalization »

RemoveDiacritics remove diacritics such as accents, umlauts, etc.

CharacterNormalize reduce or decompose characters to normal forms (e.g. ¼ to 1⁄4)

TextTranslation  ▪  Transliterate  ▪  DeleteStopwords  ▪  WordStem  ▪  ToLowerCase  ▪  ...


StringSplit split a string at spaces or other delimiters

StringCases find cases of string patterns

TextCases  ▪  TextSentences  ▪  TextWords  ▪  TextStructure

LLM-Based Operations »

LLMResourceFunction apply operations from the Wolfram Prompt Repository

LLMFunction apply operations specified by natural language descriptions

LLMExampleFunction apply operations based on examples

LLMPrompt  ▪  LLMTool  ▪  ChatEvaluate  ▪  ...

Feature Extraction

FeatureExtraction extract numerical features from text

NetModel pre-trained networks for text feature extraction

"GloVe"  ▪  "BERT"  ▪  "ELMo"  ▪  "GPT2"  ▪  ...

Content Extraction

TextSummarize automatically produce different types of summarization

FindTextualAnswer attempt to find answers to questions from text

TextContents, TextCases, TextPosition extract semantic elements in text

Text Classification

Classify classify strings based on training data or built-in classifiers

"Language"  ▪  "Profanity"  ▪  "Sentiment"  ▪  ...

LanguageIdentify identify what language a text is in

Text Clustering

FindClusters find clusters in string data

ClusteringTree  ▪  ClusteringComponents  ▪  ClusterClassify

Text Analysis »

WordCounts count of words or -grams

CharacterCounts  ▪  WordFrequency  ▪  WordData  ▪  PartOfSpeech  ▪  ...

Text Visualization

WordCloud generate a word cloud from word frequencies or weights

Snippet extract a snippet of text

Style, Highlighted style text with color, font, size, background, etc.