TokenWords is an option for Read and related functions which gives a list of token words to be used to delimit words.
The setting for TokenWords is a list of strings which are used as delimiters for words to be read.
The delimiters specified by TokenWords are themselves returned as words.
See The Mathematica Book: Section 2.11.8.
See also: WordSeparators.
THIS IS DOCUMENTATION FOR AN OBSOLETE PRODUCT.
SEE THE DOCUMENTATION CENTER
FOR THE LATEST INFORMATION.