TokenSource
- TokenSource( TextFilePath )
- Create a new TokenSource from a text file path. After creating a new token source, a valid token will be created. Calling "eat" directly after "TokenSource" would skip the first token.
- int type = type();
- The type of the current token. Positive types are applications specific. Negative types are specified by this interface specification.
- −1 End of Character Source
- −2 Read Error of Character Source
- −3 Lexical Syntax Error
- string text = text();
- The text of the current token.
- int line = startLine()
- The line of the start of the current token.
- int position = startPosition()
- The position (relative to the start of the line) of the current token.
- eat();
- Get next token
- close();
- Close this token source.