LexerCore (jain-sip-ri 1.2.213 API)

java.lang.Object
- gov.nist.core.StringTokenizer
- - gov.nist.core.LexerCore

Direct Known Subclasses:

Lexer, Lexer
```
public class LexerCore
extends StringTokenizer
```
A lexical analyzer that is used by all parsers in our implementation.

Since:

1.1

Version:

1.2

Author:

M. Ranganathan

Field Summary

Fields
Modifier and Type	Field and Description
`static int`	`ALPHA`
`static int`	`AND`
`static int`	`AT`
`static int`	`BACK_QUOTE`
`static int`	`BACKSLASH`
`static int`	`BAR`
`static int`	`COLON`
`protected Map<String,Integer>`	`currentLexer`
`protected String`	`currentLexerName`
`protected Token`	`currentMatch`
`static int`	`DIGIT`
`static int`	`DOLLAR`
`static int`	`DOT`
`static int`	`DOUBLEQUOTE`
`static int`	`END`
`static int`	`EQUALS`
`static int`	`EXCLAMATION`
`protected static ConcurrentHashMap<Integer,String>`	`globalSymbolTable`
`static int`	`GREATER_THAN`
`static int`	`HAT`
`static int`	`HT`
`static int`	`ID`
`static int`	`ID_NO_WHITESPACE`
`static int`	`L_CURLY`
`static int`	`L_SQUARE_BRACKET`
`static int`	`LESS_THAN`
`protected static ConcurrentHashMap<String,ConcurrentHashMap<String,Integer>>`	`lexerTables`
`static int`	`LPAREN`
`static int`	`MINUS`
`static int`	`NULL`
`static int`	`PERCENT`
`static int`	`PLUS`
`static int`	`POUND`
`static int`	`QUESTION`
`static int`	`QUOTE`
`static int`	`R_CURLY`
`static int`	`R_SQUARE_BRACKET`
`static int`	`RPAREN`
`static int`	`SAFE`
`static int`	`SEMICOLON`
`static int`	`SLASH`
`static int`	`SP`
`static int`	`STAR`
`static int`	`START`
`static int`	`TILDE`
`static int`	`UNDERSCORE`
`static int`	`WHITESPACE`

Fields inherited from class gov.nist.core.StringTokenizer
buffer, bufferLen, ptr, savedPtr

Constructor Summary

Constructors
Modifier Constructor and Description

protected LexerCore()

LexerCore(String lexerName, String buffer)
Initialize the lexer with a buffer.

Constructors
Modifier	Constructor and Description
`protected`	`LexerCore()`
	`LexerCore(String lexerName, String buffer)` Initialize the lexer with a buffer.

Method Summary

Methods
Modifier and Type	Method and Description
`protected void`	`addKeyword(String name, int value)`
`String`	`byteStringNoComma()` Return a substring containing no commas
`String`	`byteStringNoSemicolon()` Return a substring containing no semicolons.
`String`	`byteStringNoSlash()` Scan until you see a slash or an EOL.
`static String`	`charAsString(char ch)`
`String`	`charAsString(int nchars)` Lookahead in the inputBuffer for n chars and return as a string.
`String`	`comment()` Parse a comment string cursor is at a "(".
`void`	`consumeValidChars(char[] validChars)`
`ParseException`	`createParseException()` Create a parse exception.
`String`	`getBuffer()` Get the buffer.
`String`	`getNextId()` Get the next id.
`String`	`getNextIdNoWhiteSpace()`
`Token`	`getNextToken()`
`int`	`getPtr()` Get the read pointer.
`String`	`getRest()` Get the rest of the String
`String`	`getString(char c)` Get the sub-String until the character is encountered
`static boolean`	`isTokenChar(char c)` JvB: utility function added to validate tokens
`String`	`lookupToken(int value)`
`int`	`markInputPosition()` Mark the position for backtracking.
`Token`	`match(int tok)` Match the given token or throw an exception if no such token can be matched.
`String`	`number()` Get and consume the next number.
`String`	`peekNextId()` Peek the next id but dont move the buffer pointer forward.
`Token`	`peekNextToken()` Look ahead for one token.
`Token[]`	`peekNextToken(int ntokens)`
`String`	`quotedString()` Parse a comment string cursor is at a ".
`void`	`rewindInputPosition(int position)` Rewind the input ptr to the marked position.
`void`	`selectLexer(String lexerName)`
`void`	`SPorHT()`
`boolean`	`startsId()`
`boolean`	`startsIdNoWhiteSpace()`
`boolean`	`startsSafeToken()`
`String`	`ttoken()`
`String`	`ttokenNoWhiteSpace()`
`String`	`ttokenSafe()`

Methods inherited from class gov.nist.core.StringTokenizer
consume, consume, getLine, getLines, getNextChar, getNextToken, getSDPFieldName, hasMoreChars, isAlpha, isAlphaDigit, isDigit, isHexDigit, lookAhead, lookAhead, nextToken, peekLine

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - START
```
public static final int START
```
    See Also:
    Constant Field Values
  - END
```
public static final int END
```
    See Also:
    Constant Field Values
  - ID_NO_WHITESPACE
```
public static final int ID_NO_WHITESPACE
```
    See Also:
    Constant Field Values
  - ID
```
public static final int ID
```
    See Also:
    Constant Field Values
  - SAFE
```
public static final int SAFE
```
    See Also:
    Constant Field Values
  - WHITESPACE
```
public static final int WHITESPACE
```
    See Also:
    Constant Field Values
  - DIGIT
```
public static final int DIGIT
```
    See Also:
    Constant Field Values
  - ALPHA
```
public static final int ALPHA
```
    See Also:
    Constant Field Values
  - BACKSLASH
```
public static final int BACKSLASH
```
    See Also:
    Constant Field Values
  - QUOTE
```
public static final int QUOTE
```
    See Also:
    Constant Field Values
  - AT
```
public static final int AT
```
    See Also:
    Constant Field Values
  - SP
```
public static final int SP
```
    See Also:
    Constant Field Values
  - HT
```
public static final int HT
```
    See Also:
    Constant Field Values
  - COLON
```
public static final int COLON
```
    See Also:
    Constant Field Values
  - STAR
```
public static final int STAR
```
    See Also:
    Constant Field Values
  - DOLLAR
```
public static final int DOLLAR
```
    See Also:
    Constant Field Values
  - PLUS
```
public static final int PLUS
```
    See Also:
    Constant Field Values
  - POUND
```
public static final int POUND
```
    See Also:
    Constant Field Values
  - MINUS
```
public static final int MINUS
```
    See Also:
    Constant Field Values
  - DOUBLEQUOTE
```
public static final int DOUBLEQUOTE
```
    See Also:
    Constant Field Values
  - TILDE
```
public static final int TILDE
```
    See Also:
    Constant Field Values
  - BACK_QUOTE
```
public static final int BACK_QUOTE
```
    See Also:
    Constant Field Values
  - NULL
```
public static final int NULL
```
    See Also:
    Constant Field Values
  - EQUALS
```
public static final int EQUALS
```
    See Also:
    Constant Field Values
  - SEMICOLON
```
public static final int SEMICOLON
```
    See Also:
    Constant Field Values
  - SLASH
```
public static final int SLASH
```
    See Also:
    Constant Field Values
  - L_SQUARE_BRACKET
```
public static final int L_SQUARE_BRACKET
```
    See Also:
    Constant Field Values
  - R_SQUARE_BRACKET
```
public static final int R_SQUARE_BRACKET
```
    See Also:
    Constant Field Values
  - R_CURLY
```
public static final int R_CURLY
```
    See Also:
    Constant Field Values
  - L_CURLY
```
public static final int L_CURLY
```
    See Also:
    Constant Field Values
  - HAT
```
public static final int HAT
```
    See Also:
    Constant Field Values
  - BAR
```
public static final int BAR
```
    See Also:
    Constant Field Values
  - DOT
```
public static final int DOT
```
    See Also:
    Constant Field Values
  - EXCLAMATION
```
public static final int EXCLAMATION
```
    See Also:
    Constant Field Values
  - LPAREN
```
public static final int LPAREN
```
    See Also:
    Constant Field Values
  - RPAREN
```
public static final int RPAREN
```
    See Also:
    Constant Field Values
  - GREATER_THAN
```
public static final int GREATER_THAN
```
    See Also:
    Constant Field Values
  - LESS_THAN
```
public static final int LESS_THAN
```
    See Also:
    Constant Field Values
  - PERCENT
```
public static final int PERCENT
```
    See Also:
    Constant Field Values
  - QUESTION
```
public static final int QUESTION
```
    See Also:
    Constant Field Values
  - AND
```
public static final int AND
```
    See Also:
    Constant Field Values
  - UNDERSCORE
```
public static final int UNDERSCORE
```
    See Also:
    Constant Field Values
  - globalSymbolTable
```
protected static final ConcurrentHashMap<Integer,String> globalSymbolTable
```
  - lexerTables
```
protected static final ConcurrentHashMap<String,ConcurrentHashMap<String,Integer>> lexerTables
```
  - currentLexer
```
protected Map<String,Integer> currentLexer
```
  - currentLexerName
```
protected String currentLexerName
```
  - currentMatch
```
protected Token currentMatch
```
- Constructor Detail
  - LexerCore
```
protected LexerCore()
```
  - LexerCore
```
public LexerCore(String lexerName,
         String buffer)
```
    Initialize the lexer with a buffer.
- Method Detail
  - addKeyword
```
protected void addKeyword(String name,
              int value)
```
  - lookupToken
```
public String lookupToken(int value)
```
  - selectLexer
```
public void selectLexer(String lexerName)
```
  - peekNextId
```
public String peekNextId()
```
    Peek the next id but dont move the buffer pointer forward.
  - getNextId
```
public String getNextId()
```
    Get the next id.
  - getNextIdNoWhiteSpace
```
public String getNextIdNoWhiteSpace()
```
  - getNextToken
```
public Token getNextToken()
```
  - peekNextToken
```
public Token peekNextToken()
                    throws ParseException
```
    Look ahead for one token.
    
    Throws:
    
    ParseException
  - peekNextToken
```
public Token[] peekNextToken(int ntokens)
                      throws ParseException
```
    Throws:
    
    ParseException
  - match
```
public Token match(int tok)
            throws ParseException
```
    Match the given token or throw an exception if no such token can be matched.
    
    Throws:
    
    ParseException
  - SPorHT
```
public void SPorHT()
```
  - isTokenChar
```
public static final boolean isTokenChar(char c)
```
    JvB: utility function added to validate tokens
    
    Parameters:
    c - - character to check
    
    Returns:
    true iff character c is a valid token character as per RFC3261
    See Also:
    section 25.1: token = 1*(alphanum / "-" / "." / "!" / "%" / "*" / "_" / "+" / "`" / "'" / "~" )
  - startsId
```
public boolean startsId()
```
  - startsIdNoWhiteSpace
```
public boolean startsIdNoWhiteSpace()
```
  - startsSafeToken
```
public boolean startsSafeToken()
```
  - ttoken
```
public String ttoken()
```
  - ttokenNoWhiteSpace
```
public String ttokenNoWhiteSpace()
```
  - ttokenSafe
```
public String ttokenSafe()
```
  - consumeValidChars
```
public void consumeValidChars(char[] validChars)
```
  - quotedString
```
public String quotedString()
                    throws ParseException
```
    Parse a comment string cursor is at a ". Leave cursor at closing "
    
    Returns:
    the substring containing the quoted string excluding the closing quote.
    
    Throws:
    
    ParseException
  - comment
```
public String comment()
               throws ParseException
```
    Parse a comment string cursor is at a "(". Leave cursor at )
    
    Returns:
    the substring containing the comment excluding the closing brace.
    
    Throws:
    
    ParseException
  - byteStringNoSemicolon
```
public String byteStringNoSemicolon()
```
    Return a substring containing no semicolons.
    
    Returns:
    a substring containing no semicolons.
  - byteStringNoSlash
```
public String byteStringNoSlash()
```
    Scan until you see a slash or an EOL.
    
    Returns:
    substring containing no slash.
  - byteStringNoComma
```
public String byteStringNoComma()
```
    Return a substring containing no commas
    
    Returns:
    a substring containing no commas.
  - charAsString
```
public static String charAsString(char ch)
```
  - charAsString
```
public String charAsString(int nchars)
```
    Lookahead in the inputBuffer for n chars and return as a string. Do not consume the input.
  - number
```
public String number()
              throws ParseException
```
    Get and consume the next number.
    
    Returns:
    a substring corresponding to a number (i.e. sequence of digits).
    
    Throws:
    
    ParseException
  - markInputPosition
```
public int markInputPosition()
```
    Mark the position for backtracking.
    
    Returns:
    the current location of the pointer.
  - rewindInputPosition
```
public void rewindInputPosition(int position)
```
    Rewind the input ptr to the marked position.
    
    Parameters:
    position - - the position to rewind the parser to.
  - getRest
```
public String getRest()
```
    Get the rest of the String
    
    Returns:
    rest of the buffer.
  - getString
```
public String getString(char c)
                 throws ParseException
```
    Get the sub-String until the character is encountered
    
    Parameters:
    c - the character to match
    
    Returns:
    the substring that matches.
    
    Throws:
    
    ParseException
  - getPtr
```
public int getPtr()
```
    Get the read pointer.
  - getBuffer
```
public String getBuffer()
```
    Get the buffer.
  - createParseException
```
public ParseException createParseException()
```
    Create a parse exception.

Class LexerCore

Field Summary

Fields inherited from class gov.nist.core.StringTokenizer

Constructor Summary

Method Summary

Methods inherited from class gov.nist.core.StringTokenizer

Methods inherited from class java.lang.Object

Field Detail

START

END

ID_NO_WHITESPACE

ID

SAFE

WHITESPACE

DIGIT

ALPHA

BACKSLASH

QUOTE

AT

SP

HT

COLON

STAR

DOLLAR

PLUS

POUND

MINUS

DOUBLEQUOTE

TILDE

BACK_QUOTE

NULL

EQUALS

SEMICOLON

SLASH

L_SQUARE_BRACKET

R_SQUARE_BRACKET

R_CURLY

L_CURLY

HAT

BAR

DOT

EXCLAMATION

LPAREN

RPAREN

GREATER_THAN

LESS_THAN

PERCENT

QUESTION

AND

UNDERSCORE

globalSymbolTable

lexerTables

currentLexer

currentLexerName

currentMatch

Constructor Detail

LexerCore

LexerCore

Method Detail

addKeyword

lookupToken

selectLexer

peekNextId

getNextId

getNextIdNoWhiteSpace

getNextToken

peekNextToken

peekNextToken

match

SPorHT

isTokenChar

startsId

startsIdNoWhiteSpace

startsSafeToken

ttoken

ttokenNoWhiteSpace

ttokenSafe

consumeValidChars

quotedString

comment