TokenIterator (moe.ios.core API)

java.lang.Object
- com.ibm.icu.impl.data.TokenIterator

```
public class TokenIterator
extends Object
```
An iterator class that returns successive string tokens from some source. String tokens are, in general, separated by Pattern_White_Space in the source test. Furthermore, they may be delimited by either single or double quotes (opening and closing quotes must match). Escapes are processed using standard ICU unescaping.
2015-sep-03 TODO: Only used in com.ibm.icu.dev.test.format, move there.

Constructor Summary

Constructors
Constructor	Description
`TokenIterator(ResourceReader r)`	Construct an iterator over the tokens returned by the given ResourceReader, ignoring blank lines and comment lines (first non-blank character is '#').

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`String`	`describePosition()`	Return a string description of the position of the last line returned by readLine() or readLineSkippingComments().
`int`	`getLineNumber()`	Return the one-based line number of the line of the last token returned by next().
`String`	`next()`	Return the next token from this iterator, or null if the last token has been returned.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - TokenIterator
```
public TokenIterator(ResourceReader r)
```
    Construct an iterator over the tokens returned by the given ResourceReader, ignoring blank lines and comment lines (first non-blank character is '#'). Note that trailing comments on a line, beginning with the first unquoted '#', are recognized.
- Method Detail
  - next
```
public String next()
            throws IOException
```
    Return the next token from this iterator, or null if the last token has been returned.
    
    Throws:
    
    IOException
  - getLineNumber
```
public int getLineNumber()
```
    Return the one-based line number of the line of the last token returned by next(). Should only be called after a call to next(); otherwise the return value is undefined.
  - describePosition
```
public String describePosition()
```
    Return a string description of the position of the last line returned by readLine() or readLineSkippingComments().