Class PathLexer


  • public final class PathLexer
    extends SentenceLexer
    An implementation of a SentenceLexer that divides a sentence into words using a provided delimiter. The delimiter is a regex. Then parses the words into tokens.

    By default it tokenizes based on a "." then evaluates each word with a provided regex.

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static java.lang.String DEFAULT_EVALUATOR
      default pattern to represent a path.
      static java.lang.String DELIMITER  
    • Constructor Summary

      Constructors 
      Constructor Description
      PathLexer()
      Build a path lexer to tokenize a path.
      PathLexer​(java.lang.String delimiter)
      construct a Path lexer, remember that the delimiter is a regex, so if you want to use .
      PathLexer​(java.lang.String delimiter, java.lang.String pathPatternRegex)
      construct a Path lexer, remember that the delimiter is a regex, so if you want to use .
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      protected GResultOf<java.util.List<Token>> evaluator​(java.lang.String word, java.lang.String sentence)
      Takes in an elements such as abc or def[3] then converts it into a Token.
      java.lang.String getDeliminator()
      Return the deliminator.
      protected java.util.List<java.lang.String> tokenizer​(java.lang.String sentence)
      Takes in a string representation of a path.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • DEFAULT_EVALUATOR

        public static final java.lang.String DEFAULT_EVALUATOR
        default pattern to represent a path. Should allow most characters, even the delimiter. Although the string will be split with the delimiter so will not be seen
        See Also:
        Constant Field Values
    • Constructor Detail

      • PathLexer

        public PathLexer()
        Build a path lexer to tokenize a path.
      • PathLexer

        public PathLexer​(java.lang.String delimiter)
        construct a Path lexer, remember that the delimiter is a regex, so if you want to use . you need to escape it. ".".
        Parameters:
        delimiter - the character to split the sentence
      • PathLexer

        public PathLexer​(java.lang.String delimiter,
                         java.lang.String pathPatternRegex)
        construct a Path lexer, remember that the delimiter is a regex, so if you want to use . you need to escape it. "."
        Parameters:
        delimiter - the character to split the sentence
        pathPatternRegex - a regex with capture groups to decide what kind of token this is. The regex should have a capture group name = name of the element array = if this element is an array index = the index for the array
    • Method Detail

      • getDeliminator

        public java.lang.String getDeliminator()
        Description copied from class: SentenceLexer
        Return the deliminator.
        Specified by:
        getDeliminator in class SentenceLexer
        Returns:
        the deliminator
      • tokenizer

        protected java.util.List<java.lang.String> tokenizer​(java.lang.String sentence)
        Description copied from class: SentenceLexer
        Takes in a string representation of a path. Ie abc.def or ABC_DEF then returns each element separated by the delimiter
        Specified by:
        tokenizer in class SentenceLexer
        Parameters:
        sentence - the sentence to tokenize
        Returns:
        list of tokenized strings from sentance.
      • evaluator

        protected GResultOf<java.util.List<Token>> evaluator​(java.lang.String word,
                                                             java.lang.String sentence)
        Description copied from class: SentenceLexer
        Takes in an elements such as abc or def[3] then converts it into a Token.
        Specified by:
        evaluator in class SentenceLexer
        Parameters:
        word - The First string in the method params is the word in the string we are evaluating
        sentence - second string in the method params is the path the word is in.
        Returns:
        GResultOf list of tokens from a word.