com.aaronicsubstances.code.augmentor.core.util

Class SourceCodeTokenizer

java.lang.Object

com.aaronicsubstances.code.augmentor.core.util.SourceCodeTokenizer

public class SourceCodeTokenizer
extends java.lang.Object

Tokenizes source code files into token lines.

Constructor Summary

Constructors

Constructor and Description

SourceCodeTokenizer(java.util.List<java.lang.String> genCodeStartDirectives, java.util.List<java.lang.String> genCodeEndDirectives, java.util.List<java.lang.String> embeddedStringDirectives, java.util.List<java.lang.String> embeddedJsonDirectives, java.util.List<java.lang.String> skipCodeStartDirectives, java.util.List<java.lang.String> skipCodeEndDirectives, java.util.List<java.util.List<java.lang.String>> augCodeDirectiveSets, java.util.List<java.lang.String> inlineGenCodeDirectives, java.util.List<java.lang.String> nestedLevelStartMarkers, java.util.List<java.lang.String> nestedLevelEndMarkers) Constructs an instance for tokenizing with a given set of directives.

Method Summary

Modifier and Type	Method and Description
java.util.List<Token>	tokenizeSource(java.lang.String source) Tokenizes source code file into tokens.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SourceCodeTokenizer

public SourceCodeTokenizer(java.util.List<java.lang.String> genCodeStartDirectives,
                           java.util.List<java.lang.String> genCodeEndDirectives,
                           java.util.List<java.lang.String> embeddedStringDirectives,
                           java.util.List<java.lang.String> embeddedJsonDirectives,
                           java.util.List<java.lang.String> skipCodeStartDirectives,
                           java.util.List<java.lang.String> skipCodeEndDirectives,
                           java.util.List<java.util.List<java.lang.String>> augCodeDirectiveSets,
                           java.util.List<java.lang.String> inlineGenCodeDirectives,
                           java.util.List<java.lang.String> nestedLevelStartMarkers,
                           java.util.List<java.lang.String> nestedLevelEndMarkers)

Constructs an instance for tokenizing with a given set of directives. Null and empty string directives and directives consisting of only whitespace will be not be used, and will be skipped.

Parameters:

genCodeStartDirectives -

genCodeEndDirectives -

embeddedStringDirectives -

embeddedJsonDirectives -

skipCodeStartDirectives -

skipCodeEndDirectives -

augCodeDirectiveSets -

inlineGenCodeDirectives -

nestedLevelStartMarkers -

nestedLevelEndMarkers -

Method Detail

tokenizeSource

public java.util.List<Token> tokenizeSource(java.lang.String source)

Tokenizes source code file into tokens.

Parameters:

source - contents of source code file.

Returns:

tokens.