SyntaxTokenizer (J/Link 4.1 API Documentation)

Overview

Package

Class

Use

Tree

Deprecated

Index

Help

PREV CLASS NEXT CLASS

FRAMES NO FRAMES

SUMMARY: NESTED | FIELD | CONSTR | METHOD

DETAIL: FIELD | CONSTR | METHOD

com.wolfram.jlink.ui
Class SyntaxTokenizer

java.lang.Object
  com.wolfram.jlink.ui.SyntaxTokenizer

public class SyntaxTokenizer
extends java.lang.Object
extends java.lang.Object

A utility class that can break up Mathematica code into 4 syntax classes: strings, comments, symbols, and normal (meaning everything else). This class is used by MathSessionPane to implement its syntax coloring feature, but you can use it directly in your own programs.

To use a SyntaxTokenizer, construct one and then call its setText() method, supplying the Mathematica input you want tokenized. You then call getNextRecord() repeatedly to get SyntaxRecords, which tell you the type of syntax element and the length in characters.

This process is very fast. You can iterate through 100,000 characters of Mathematica code in a small fraction of a second

Here is some sample code that demonstrates how to use a SyntaxTokenizer:

        String input = "some Mathematica code here";
        SyntaxTokenizer tok = new SyntaxTokenizer();
        tok.setText(input);
        while(tok.hasMoreRecords()) {
                SyntaxTokenizer.SyntaxRecord rec = tok.getNextRecord();
                System.out.println("type: " + rec.type);
                System.out.println("text: " + input.substring(rec.start, rec.start + rec.length));
        }

Since:: 2.0
See Also:: MathSessionPane

Nested Class Summary

class SyntaxTokenizer.SyntaxRecord
          A simple class the encapsulates information about a syntax element.

Field Summary

static int COMMENT
          A syntax type that corresponds to a Mathematica comment.

static int NORMAL
          A syntax type that consists of everything other than STRING, COMMENT, or SYMBOL.

static int STRING
          A syntax type that corresponds to a literal string.

static int SYMBOL
          A syntax type that corresponds to a Mathematica symbol.

Constructor Summary

SyntaxTokenizer()


Method Summary

SyntaxTokenizer.SyntaxRecord getNextRecord()
          Gets the next SyntaxRecord specifying the type of the element (SYMBOL, STRING, COMMENT or NORMAL), its start position, and length.

boolean hasMoreRecords()
          Returns true or false to indicate whether there are any more records left in the text (i.e., whether we have come to the end of the input).

void reset()
          Resets the state of the tokenizer so that the next call to getNextRecord() will retrieve the first record in the text.

void setText(java.lang.String text)
          Sets the Mathematica input text to tokenize.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

NORMAL

public static final int NORMAL

A syntax type that consists of everything other than STRING, COMMENT, or SYMBOL.

See Also:: Constant Field Values

STRING

public static final int STRING

A syntax type that corresponds to a literal string.

See Also:: Constant Field Values

COMMENT

public static final int COMMENT

A syntax type that corresponds to a Mathematica comment.

See Also:: Constant Field Values

SYMBOL

public static final int SYMBOL

A syntax type that corresponds to a Mathematica symbol.

See Also:: Constant Field Values

Constructor Detail