WOLFRAM

Wolfram Language & System Documentation Center

CodeTokenize["code"]

tokenizes a string of WL code returning a list of tokens.

CodeTokenize[File[src]]

tokenizes a file of WL code returning a list of tokens.

Details and Options
Details and Options Details and Options
Examples  
Options  
SourceConvention  
Possible Issues  
Related Guides
CodeParser`
CodeParser`

CodeTokenize

CodeTokenize["code"]

tokenizes a string of WL code returning a list of tokens.

CodeTokenize[File[src]]

tokenizes a file of WL code returning a list of tokens.

Details and Options

  • CodeTokenize has these options:
  • SourceConvention "LineColumn"convention to use for Source metadata

Examples

open all close all

Options  (1)

SourceConvention  (1)

Using "LineColumn" convention:

Using "SourceCharacterIndex" convention:

Possible Issues  (1)

A string representing a file path is treated as code.

File paths must use the File wrapper.