This is documentation for Mathematica 7, which was
based on an earlier version of the Wolfram Language.
View current documentation (Version 11.1)

TokenWords

TokenWords
is an option for Read and related functions which gives a list of token words to be used to delimit words.
  • The setting for TokenWords is a list of strings which are used as delimiters for words to be read.
  • The delimiters specified by TokenWords are themselves returned as words.
A text file:
Read the file:
Read the file using token words to delimit words in the text:
A text file:
Read the file:
In[2]:=
Click for copyable input
Out[2]=
Read the file using token words to delimit words in the text:
In[3]:=
Click for copyable input
Out[3]=
New in 2