public class Token
extends java.lang.Object
SourceCodeTokenizer.
A token is a line with data associated with it for classifying it.
| Modifier and Type | Field and Description |
|---|---|
int |
augCodeSpecIndex
Applies only to augmenting code token lines to indicate the
index of the directive set of which the directive of the augmenting
code line is part.
|
static int |
DIRECTIVE_TYPE_AUG_CODE
Indicates augmenting code section line other than
embedded string or embedded JSON line.
|
static int |
DIRECTIVE_TYPE_EMB_JSON
Indicates embedded JSON line.
|
static int |
DIRECTIVE_TYPE_EMB_STRING
Indicates embedded string line.
|
static int |
DIRECTIVE_TYPE_SKIP_CODE_END
Indicates that a token line ends a skip code section or
a generated code section.
|
static int |
DIRECTIVE_TYPE_SKIP_CODE_START
Indicates that a token line starts a skip code section,
starts a generated code section, or is an inline generated
code section line.
|
java.lang.String |
directiveContent
Stores substring of line after directive text, after
nested level marker, and before ending newline.
|
java.lang.String |
directiveMarker
If token line has a directive as prefix, this field stores
the text of that directive.
|
int |
endPos
offset after end of token line.
|
java.lang.String |
indent
Leading whitespace indent of token line.
|
int |
index
Used to store index of token in tokens parsed from a file.
|
boolean |
isGeneratedCodeMarker
Identifies use of generated code directive in a
skip code directive token line.
|
boolean |
isInlineGeneratedCodeMarker
Identifies use of inline generated code directive in a
skip code directive token line.
|
int |
lineNumber
Token line number.
|
java.lang.String |
nestedLevelEndMarker
Stores any nested level end marker found in token line starting
augmenting code section.
|
int |
nestedLevelNumber
Available for use during prepare stage of Code Augmentor to store
nested level numbers of augmenting code token lines.
|
java.lang.String |
nestedLevelStartMarker
Stores any nested level start marker found in token line starting
augmenting code section.
|
java.lang.String |
newline
Ending newline of token line or null if line is the last line
of a file which doesn't end with a newline.
|
int |
startPos
offset of token line
|
java.lang.String |
text
full text of token line including any ending newline.
|
int |
type
type of token line
|
static int |
TYPE_BLANK
Indicates line consisting of only whitespace.
|
static int |
TYPE_OTHER
Indicates line which does not start with a directive, and
has non whitespace character.
|
| Constructor and Description |
|---|
Token()
Used for JSON serialization.
|
Token(int type)
Constructs token with required type of the token
|
| Modifier and Type | Method and Description |
|---|---|
boolean |
equals(java.lang.Object obj) |
int |
hashCode() |
java.lang.String |
toString() |
public static final int DIRECTIVE_TYPE_SKIP_CODE_START
public static final int DIRECTIVE_TYPE_SKIP_CODE_END
public static final int DIRECTIVE_TYPE_EMB_STRING
public static final int DIRECTIVE_TYPE_EMB_JSON
public static final int DIRECTIVE_TYPE_AUG_CODE
public static final int TYPE_BLANK
public static final int TYPE_OTHER
public int type
public java.lang.String text
public int startPos
public int endPos
public int lineNumber
public int index
public int augCodeSpecIndex
public java.lang.String indent
public java.lang.String directiveMarker
public java.lang.String directiveContent
public java.lang.String newline
public boolean isGeneratedCodeMarker
public boolean isInlineGeneratedCodeMarker
public java.lang.String nestedLevelStartMarker
public java.lang.String nestedLevelEndMarker
public int nestedLevelNumber